I have always thought it would be neat to be able to navigate and edit source files on a remote system through a web browser. This is not to say that there are no tools in existence today for remotely accessing file systems and files. There may even be some tools that provide similar functionality for web browsers. Nevertheless, this seemed like an interesting challenge.
Snorkel Developer's Guide
Using the Code
The diagram above illustrates the project's design. The solution uses two components: a server component for accepting incoming requests, and a plug-in for processing requests for viewing and editing source files.
We begin with the server component.
18 main (int argc, char *argv)
20 int i = 1;
21 int http_port = 0;
22 int https_port = 0;
23 char *pszIndex = 0;
27 snorkel_obj_t http = 0;
120 if (snorkel_init () != SNORKEL_SUCCESS)
122 perror ("could not initialize snorkel\n");
123 exit (1);
131 http = snorkel_obj_create (snorkel_obj_server,
132 2, pszIndex);
133 if (!http)
135 perror ("could not create server object!\n");
136 exit (1);
147 != SNORKEL_SUCCESS)
149 fprintf (stderr,
150 "error encountered setting bubbles!");
151 exit (1);
154 snorkel_obj_set (http, snorkel_attrib_show_dir, 1);
160 if (http_port)
162 if (snorkel_obj_set (http,
164 http_port, 0) != SNORKEL_SUCCESS)
166 fprintf (stderr,
167 "error could not add listener for port %d\n",
169 exit (1);
230 fprintf (stderr,
231 "\n\n[HTTPS] starting embedded server...\n");
239 if (snorkel_obj_start (http) != SNORKEL_SUCCESS)
241 perror ("could not start server\n");
242 snorkel_obj_destroy (http);
243 if (logobj)
244 snorkel_obj_destroy (logobj);
245 exit (1);
256 fprintf (stderr,
257 "\n[HTTP] started.\n\n--hit enter to terminate--\n");
258 fgets (szExit, sizeof (szExit), stdin);
main, we initialize the Snorkel API by making a call to
Snorkel_init must be called before calling any Snorkel API. Next, we create an HTTP object by invoking the
snorkel_obj_create function. The
snorkel_obj_create function provides object creation for all Snorkel objects. Unlike C++ and Java, Snorkel objects are not derived from classes. They are more like Windows handles, encapsulations of related data obfuscated by a void pointer. The function takes three parameters: the object type (
snorkel_obj_server), the number of threads to create (two), and a pointer to a null terminated string containing the fully qualified path to the directory that we want to export.
On line 146, we identify the plug-in directory. By passing a
NULL pointer for the third argument, we instruct the API to use the default plug-in directory: (current directory)/bubbles.
Plug-ins, known as bubbles in the Snorkel API, are self-contained runtime components that export functions with URI mappings for processing HTTP requests. In this example, server logic for processing HTTP-GETs and HTTP-POSTs related to file editing and viewing reside in a plug-in (bubble). Placing the functionality into a bubble allows for reuse by other Snorkel based embedded servers.
As with any web-server, an index file is required to resolve the initial HTTP request sent by a browser. By default, an error occurs if the index directory, the directory provided on line 131, does not contain an index file (index.html). The Snorkel API provides functionality for directory navigation through a browser, if an index file is not present; however, by default, it is disabled. On line 154, we enable browser based directory navigation by calling
snorkel_obj_set with the server attribute
snorkel_attrib_show_dir. Since we are not providing the server with the location of an index file, it will display the directory listing within the browser instead of producing the default error, page not found.
The directory listing provided by the Snorkel runtime provides file navigation from the root directory (the directory identified on line131) and all of its sub-directories. Directory listings contain links for both files and sub-directories. Names delimited by a beginning and ending bracket denote directory links. The listing headers (Name, Size, and Date Modified) are also links; selecting them sorts the directory listing based on the header type. For example, selecting the Name header sorts the listing based on the file name.
Before we can issue the instruction to start the server, we need to define a listener. Snorkel listeners are objects that process TCP/IP-based protocol requests over user defined ports. A Snorkel server object can support multiple listeners. Listeners can listen for and process requests from HTTP, HTTPS, or proprietary protocol based clients. On lines 160-170, we define a listener assigned to the port
http_port, a value obtained from a command line option.
Finally, on line 239, we issue a call to
snorkel_obj_start to start the embedded web-server. The
snorkel_obj_start API starts the server as a separate thread and returns control back to the calling routine. To prevent the application from exiting, we use the
fgets command to wait for input from the end-user to determine when to exit. The entire listing for the program can be found in the "c" directory of the attached bundle in the file file_server.c.
After defining the server component, we next define the plug-in - our Snorkel bubble.
After loading a plug-in into memory, the Snorkel runtime checks for the function
bubble_main. If the function exists, the runtime calls the function passing it the associated server object, the object we created in the server's main. In
bubble_main, we issue calls to the API
snorkel_obj_set with the attribute
snorkel_attrib_mime to associate each file type with a content type and a callback function (
view_uri). We also associate any URI containing the ending string "update.html" with the function
save_file. When the embedded server encounters a request for an associated file type, that is, a file type associated with a mapped function, it calls the function, passing an HTTP request object, a connection object, and a the URL associated with the requested URI. In this solution, the runtime calls the function
view_uri for any file containing a matching-mapped file type.
742 byte_t SNORKEL_EXPORT
743 bubble_main (snorkel_obj_t server)
745 int i = 0;
747 snorkel_obj_set (server, snorkel_attrib_mime, "c",
751 snorkel_obj_set (server, snorkel_attrib_mime, "cpp",
755 snorkel_obj_set (server, snorkel_attrib_mime, "h";,
815 snorkel_obj_set (server,
816 snorkel_attrib_uri, POST,
817 "*update.html", encodingtype_text,
831 return 1;
In the function
view_uri, we test for the presence of a query-string containing either edit or download. We use the query-string to determine how an end-user wishes to process a mapped file type. A query-string is any string appended to a URI proceeded by a '?'. For example, if the HTTP request contained the URI "http://localhost/c/source.c?edit", the query string is "edit". The Snorkel runtime treats query-strings as a header element, and stores them in the HTTP-header variable "
QUERY". On lines 643-645, we use the API function
snorkel_obj_get with the attribute
snorkel_attrib_header to retrieve the query-string value from the HTTP request header.
593 call_status_t SNORKEL_EXPORT
594 view_uri (snorkel_obj_t http,
595 snorkel_obj_t connection, char *pszurl)
643 if (snorkel_obj_get
644 (http, snorkel_attrib_header, "QUERY", szquery,
645 (int) sizeof (szquery)) == SNORKEL_SUCCESS)
647 if (strcmp (szquery, "edit") == 0)
648 return edit_page (http, connection, pszurl);
649 else if (strcmp (szquery, "download") == 0)
651 if (snorkel_file_stream
652 (connection, pszurl, 0,
653 SNORKEL_BINARY) == SNORKEL_ERROR)
654 return HTTP_ERROR;
655 return HTTP_SUCCESS;
738 return SNORKEL_SUCCESS;
If the query-string equals "edit", we send the HTTP request object, connection object, and the URL to the function
408 edit_page (snorkel_obj_t http,
409 snorkel_obj_t connection, char *pszurl)
443 if (stat (pszurl, &stf) != 0)
445 ERROR_STRING ("resource could not be located\r\n");
447 strftime(sztime,sizeof(sztime),"%m/%d/%y %I:%M:%S %p",
448 localtime (&stf.st_mtime));
450 snorkel_obj_get (http, snorkel_attrib_uri, szuri,
451 (int) sizeof (szuri));
452 pszfile = strrchr (szuri, '/');
455 if (snorkel_printf(connection,header,szuri, sztime) ==
457 return HTTP_ERROR;
460 if (snorkel_file_stream
461 (connection, pszurl, 0,
462 SNORKEL_UUENCODE) == SNORKEL_ERROR)
463 return HTTP_ERROR;
465 if (snorkel_printf (connection, footer, pszfile) ==
467 return HTTP_ERROR;
469 return HTTP_SUCCESS;
In the function
edit_page, we send back an HTTP reply as an HTML form containing the associated file content in an edit field along with the associated filename in a non-editable field. To write the reply, we use a combination of the functions
snorkel_file_stream. The function
snorkel_printf works like the C
fprintf function, using the connection object as an opened stream. We use the function to write the HTML header and footer. The
snorkel_file_stream function streams the source file referenced by the URL to the client uuencoded.
We associated the update button, Submit, with the function
save_file on lines 815-818 in
bubble_main. If the user selects the update button from an
edit_page-form, the server calls the exported function
save_file to save file modifications to the associated URL.
472 call_status_t SNORKEL_EXPORT
473 save_file (snorkel_obj_t http, snorkel_obj_t con)
490 if (snorkel_obj_get
491 (http, snorkel_attrib_local_url_path, szurl_file,
492 sizeof (szurl_file)) != SNORKEL_SUCCESS)
493 return HTTP_ERROR;
496 if (snorkel_obj_get
497 (http, snorkel_attrib_uri_path, szuri_file,
498 sizeof (szuri_file)) != SNORKEL_SUCCESS)
499 return HTTP_ERROR;
501 szuri_path = 0;
502 strcat (szuri_path, szuri_file);
504 if (snorkel_obj_get
505 (http, snorkel_attrib_post, "filename", szfile,
506 sizeof (szfile)) == SNORKEL_ERROR)
507 return HTTP_ERROR;
510 if (snorkel_obj_get
511 (http, snorkel_attrib_post_ref, "contents", &psz,
512 &cbpsz) == SNORKEL_ERROR)
513 return HTTP_ERROR;
515 #if defined(WIN32) || defined(WIN64)
516 strcat (szurl_file, "\\");
518 strcat (szurl_file, "/");
520 if (szuri_file[strlen (szuri_file) - 1] != '/')
521 strcat (szuri_file, "/");
523 strcat (szurl_file, szfile);
524 strcat (szuri_file, szfile);
526 fd = fopen (szurl_file, "wb");
527 if (!fd)
530 ERROR_STRING ("The file could not be saved!\r\n");
533 ptr = psz;
534 while (ptr)
536 char *temp = ptr;
537 ptr = strstr (ptr, "\r\n");
538 if (ptr && ptr != temp)
540 *ptr = 0;
541 if ( fprintf(fd, "%s\n";, temp) < 0)
543 fclose (fd);
545 ("I/O error encountered updating file.\r\n");
547 ptr += 2;
549 else if (ptr && ptr == temp)
551 if (fprintf (fd, "\n") < 0 )
553 fclose (fd);
556 ("I/O error encountered updating file.\r\n");
558 ptr += 2;
560 else if (temp && strlen (temp) > 0)
562 if (fprintf (fd, "%s\n", temp) < 0)
564 fclose (fd);
567 ("I/O error encountered updating file.\r\n");
573 fclose (fd);
574 snorkel_printf (con, pszsuccess,
575 szuri_file, i);
576 return HTTP_SUCCESS;
To write the file, the
save_file function gets the filename stored in the non-editable field of the
edit_page-form and the modified file content using the
snorkel_obj_get API with the
snorkel_attrib_post_ref attributes. All data for posted-forms is stored in a table accessible by these attributes. The
snorkel_attrib_post attribute obtains HTTP-POST variable values by copying the variable value into a provided buffer, whereas the
snorkel_attrib_post_ref returns a pointer to the data stored in an HTTP-POST variable. We use the latter to eliminate the need for the allocation of a buffer large enough to store the modified file's content. To save the file, the
save_file function opens the file provided in the variable
filename and writes the content contained in the variable
If the query-string value, obtained by
view_uri, is "download",
view_uri streams the file referenced by the URI back to the requesting client line by line, insuring proper line termination. This might seem inefficient, but thanks to the Snorkel runtime, it is not. The Snorkel runtime automatically buffers small I/O sends to reduce the number of calls to the Socket layer.
Finally, if the URI does not contain a query-string, the
view_uri checks the extension and formats the HTML response based on the file type.
The source files for this project (file_server.c and file_server_plugin.c) are located in the "c" directory of the attached bundle.
Running the Server
To run the server, extract the content of the attached bundle.
LD_LIBRARY_PATH to include the directory deployment_directory/lib/Linux.
- Change directories to deployment_dir/bin/Linux.
- Enter the command "fsrv -p 8080 -i deployment_directory".
- Change directories to deployment_directory\bin\wintel.
- Enter the command "fsrv -p 8080 -i deployment_directory".
Start a browser and point it at http://server_hostname:8080.
I am providing a limited version of the Snorkel runtime library used in this project to CodeProject members free of charge. The provided SDK includes binaries for Linux, SunOS, and Windows platforms. It also includes examples and a developer's guide. Even though Snorkel supports SSL, I have removed SSL versions of the runtime libraries due to US trade laws. The developer's guide is located in the doc folder of the attached bundle, you may want to read the guide prior to playing around with this project. If you have any questions and or additional interests in the provided SDK, feel free to contact me at firstname.lastname@example.org.
- March 30, 2010 - Made some grammatical corrections and included a direct link to the Snorkel Developer's Guide.
- April 08, 2010 - Added the Windows 2K version of Snorkel runtime. Note: the Windows 2000 runtime does not leverage thread affinity since the supporting APIs are not present in the OS implementation.
- April 09, 2010 - Made corrections for the Windows 2K version, per request.
- April, 09 2010 - Made some corrections to page 24 of the Developer's Guide.
- April 14, 2010 - Updated Snorkel runtime. The update fixes a
CLOSE_WAIT issue, which can occur when a connection is abruptly lost. Also exposed both the linger and timeout attributes for listeners, see modified version of file_server.c and/or the Developer's Guide for how to use. Updated the Developer's Guide in the bundle.
- April 22, 2010 - Due to an error in the SDK build process, in the last update, the Windows 2K (snorkel32_2k.dll) version of the runtime became the de facto version of the runtime. This was because the link libraries for both the Windows 2K and non-Windows 2K versions for the Wintel platform shared the same library name, snorkel32.lib. In this update for the SDK, the Windows 2K version correctly uses the linked library name, snorkel32_2k.lib. Note, this still allows snorkel32_2k.dll to be substituted for snorkel32.dll by altering its name.
- May 18, 2010 - Corrected an issue regarding binary data streaming and MIME-URL callbacks. The defect did not affect callbacks that streamed non-binary content such as HTML, XML, text, etc... Added a network performance enhancement to the Snorkel runtime to improve server response under heavy load conditions.
Note: the May 18, 2010 update includes an extension of the file server bubble to include GNU Plot files. The modification facilitates data plotting in browsers. Please hold off on using the new functionality and wait for the accompanying article. I will not address questions regarding the new functionality on this page.
- June 16, 2010 - updated Snorkel runtime to 1.0, and added additional command line options to access new functionality.
Snorkel 1.0 includes:
- Support for keep-alive.
- Support for zero-copy (sendfile).
- Exposed thread governor overload -- now users can create more handlers than there are cores.
- Minor bug fixes.
- Added built-in page to display information about embedded server. To access page, append root URI with /about or /snorkel.
- June 21, 2010
I normally don't post updates to my libraries this fast, but this time, I could not resist the temptation. Build 1.0.1 received a significant performance boost over the weekend, on the Windows side, thanks to changes in how file system information is cached. The changes significantly improved requests per second and average transfer rate by reducing I/O blocking. When bench marked against other web-servers, using Apache's ab test, the performance differences were significant enough to merit this early update.
- June 21, 2010
Noticed and corrected defects on UNIXes.
- June 25, 2010 -- Snorkel 1.0.2 update
- Fixed a minor thread-heap allocator defect for allocations pushed outside of thread-heap.
- Added thread-heap integrity check and auto-repair features.
- Completed testing and enabled additional performance enhancements that were introduced but disabled for 1.0.1.
Note to adopters of Snorkel: Snorkel is heavily tested on a daily basis, and bugs are often detected and fixed before they are detected in the field. Until there is an official site for Snorkel updates, I will keep the version bundled with this article and other articles that use the library up to date with the latest version of the API.
- July 8, 2010 -- Snorkel 1.0.4 update
- Corrected file mismatch between runtime library files in the bin directory and the lib directory for Wintel.
- I added ability to toggle off/on the size field transmitted by
snorkel_printf for non-HTTP streams. To enable or disable the feature, which is on by default for non-HTTP streams, use the following command
snorkel_obj_set (snorkel_obj_t non_http_stream, snorkel_attrib_cbprintf, int state (enabled=1,disabled=0)).
- Added email function. Syntax:
snorkel_smtp_message (char *smtp_server, int port, char *from, char *to, char *format_string, arg1, arg2,...argN). Note: the function works just like
- Exposed a few more attributes.
- Minor bug fixes.
- July 21, 2010 -- Snorkel 1.0.5 update
- January 6, 2011 – Snorkel 126.96.36.199 update (still free)
- New improved API
- Faster performance
- Added new APIs Aqua (Service SDK) Sailfish (C/C++ application server)
- Platform support MAC OSX, SunOS, Debian Linux, Windows
- New download site: http://snorkelembedded.webs.com
Currently I am a Software Architect for Compuware specializing in software security
Languages: C/C++, FORTRAN, COBOL, Java
Libraries: OpenGL, MFC, X11 (X-Windows), WIN32
Platform Experience: AIX, HP-UX, SunOS, Open VMS, AS400, OSF, AIX, SGI, Linux, Windows CE, Windows
I have a strong background in 3D graphics programming, TCP/IP development, threading, cross platform development, encryption and secured communications, and system level programming.
In the early years before OpenGL made its way to the PC world, I authored one of the first high performance double buffered 3D graphics engines that supported both DOS and Windows providing real time 3D graphics for Engineering Technology Associate’s Finite Element Model Builder, FEMB. You can see their products at www.eta.com.
I also hold a US patent http://www.patentstorm.us/patents/6894690.html