diff --git a/README.md b/README.md index 17b0ffa5..cd66729a 100644 --- a/README.md +++ b/README.md @@ -269,6 +269,7 @@ For example, if your connection limit is “1”, a browser may open a first con * `FILE_UPLOAD_MEMORY_AND_DISK`: The content of the file is stored in memory and on the file system. * _.file_upload_dir(**const std::string&** file_upload_dir):_ Specifies the directory to store all uploaded files. Default value is `/tmp`. * _.generate_random_filename_on_upload() and .no_generate_random_filename_on_upload():_ Enables/Disables the library to generate a unique and unused filename to store the uploaded file to. Otherwise the actually uploaded file name is used. `off` by default. +* _.file_cleanup_callback(**file_cleanup_callback_ptr** callback):_ Sets a callback function to control what happens to uploaded files when the request completes. By default (when no callback is set), all uploaded files are automatically deleted. The callback signature is `bool(const std::string& key, const std::string& filename, const http::file_info& info)` where `key` is the form field name, `filename` is the original uploaded filename, and `info` contains file metadata including the filesystem path. Return `true` to delete the file (default behavior) or `false` to keep it (e.g., after moving it to permanent storage). If the callback throws an exception, the file will be deleted as a safety measure. * _.deferred()_ and _.no_deferred():_ Enables/Disables the ability for the server to suspend and resume connections. Simply put, it enables/disables the ability to use `deferred_response`. Read more [here](#building-responses-to-requests). `on` by default. * _.single_resource() and .no_single_resource:_ Sets or unsets the server in single resource mode. This limits all endpoints to be served from a single resource. The resultant is that the webserver will process the request matching to the endpoint skipping any complex semantic. Because of this, the option is incompatible with `regex_checking` and requires the resource to be registered against an empty endpoint or the root endpoint (`"/"`). The resource will also have to be registered as family. (For more information on resource registration, read more [here](#registering-resources)). `off` by default. @@ -718,6 +719,37 @@ Details on the `http::file_info` structure. * _**const std::string** get_content_type() **const**:_ Returns the content type of the file uploaded through the HTTP request. * _**const std::string** get_transfer_encoding() **const**:_ Returns the transfer encoding of the file uploaded through the HTTP request. +#### Example of keeping uploaded files +By default, uploaded files are automatically deleted when the request completes. To keep files (e.g., move them to permanent storage), use the `file_cleanup_callback`: + +```cpp +#include +#include + +using namespace httpserver; + +int main() { + webserver ws = create_webserver(8080) + .file_upload_target(FILE_UPLOAD_DISK_ONLY) + .file_upload_dir("/tmp/uploads") + .file_cleanup_callback([](const std::string& key, + const std::string& filename, + const http::file_info& info) { + // Move file to permanent storage + std::string dest = "/var/uploads/" + filename; + std::rename(info.get_file_system_file_name().c_str(), dest.c_str()); + return false; // Don't delete - we moved it + }); + + // ... register resources and start server +} +``` +To test file uploads, you can run the following command from a terminal: + + curl -XPOST -F "file=@/path/to/your/file.txt" 'http://localhost:8080/upload' + +You can also check this example on [github](https://github.com/etr/libhttpserver/blob/master/examples/file_upload_with_callback.cpp). + Details on the `http_arg_value` structure. * _**std::string_view** get_flat_value() **const**:_ Returns only the first value provided for the key. diff --git a/examples/Makefile.am b/examples/Makefile.am index cf838c30..14a228c1 100644 --- a/examples/Makefile.am +++ b/examples/Makefile.am @@ -19,7 +19,7 @@ LDADD = $(top_builddir)/src/libhttpserver.la AM_CPPFLAGS = -I$(top_srcdir)/src -I$(top_srcdir)/src/httpserver/ METASOURCES = AUTO -noinst_PROGRAMS = hello_world service minimal_hello_world custom_error allowing_disallowing_methods handlers hello_with_get_arg setting_headers custom_access_log basic_authentication digest_authentication minimal_https minimal_file_response minimal_deferred url_registration minimal_ip_ban benchmark_select benchmark_threads benchmark_nodelay deferred_with_accumulator file_upload +noinst_PROGRAMS = hello_world service minimal_hello_world custom_error allowing_disallowing_methods handlers hello_with_get_arg setting_headers custom_access_log basic_authentication digest_authentication minimal_https minimal_file_response minimal_deferred url_registration minimal_ip_ban benchmark_select benchmark_threads benchmark_nodelay deferred_with_accumulator file_upload file_upload_with_callback hello_world_SOURCES = hello_world.cpp service_SOURCES = service.cpp @@ -42,6 +42,7 @@ benchmark_select_SOURCES = benchmark_select.cpp benchmark_threads_SOURCES = benchmark_threads.cpp benchmark_nodelay_SOURCES = benchmark_nodelay.cpp file_upload_SOURCES = file_upload.cpp +file_upload_with_callback_SOURCES = file_upload_with_callback.cpp if HAVE_GNUTLS LDADD += -lgnutls diff --git a/examples/file_upload_with_callback.cpp b/examples/file_upload_with_callback.cpp new file mode 100644 index 00000000..edc5338f --- /dev/null +++ b/examples/file_upload_with_callback.cpp @@ -0,0 +1,111 @@ +/* + This file is part of libhttpserver + Copyright (C) 2011, 2012, 2013, 2014, 2015 Sebastiano Merlino + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 + USA +*/ + +#include +#include +#include +#include + +#include + +class file_upload_resource : public httpserver::http_resource { + public: + std::shared_ptr render_GET(const httpserver::http_request&) { + std::string get_response = "\n"; + get_response += " \n"; + get_response += "

File Upload with Cleanup Callback Demo

\n"; + get_response += "

Uploaded files will be moved to the permanent directory.

\n"; + get_response += "
\n"; + get_response += " \n"; + get_response += "

\n"; + get_response += " \n"; + get_response += "
\n"; + get_response += " \n"; + get_response += "\n"; + + return std::shared_ptr(new httpserver::string_response(get_response, 200, "text/html")); + } + + std::shared_ptr render_POST(const httpserver::http_request& req) { + std::string post_response = "\n"; + post_response += "\n"; + post_response += "

Upload Complete

\n"; + post_response += "

Files have been moved to permanent storage:

\n"; + post_response += "
    \n"; + + for (auto &file_key : req.get_files()) { + for (auto &files : file_key.second) { + post_response += "
  • " + files.first + " (" + + std::to_string(files.second.get_file_size()) + " bytes)
    • \n"; + } + } + + post_response += "
\n"; + post_response += " Upload more\n"; + post_response += "\n"; + return std::shared_ptr(new httpserver::string_response(post_response, 201, "text/html")); + } +}; + +int main(int argc, char** argv) { + if (3 != argc) { + std::cout << "Usage: file_upload_with_callback " << std::endl; + std::cout << std::endl; + std::cout << " temp_dir: directory for temporary upload storage" << std::endl; + std::cout << " permanent_dir: directory where files will be moved after upload" << std::endl; + return -1; + } + + std::string temp_dir = argv[1]; + std::string permanent_dir = argv[2]; + + std::cout << "Starting file upload server on port 8080..." << std::endl; + std::cout << " Temporary directory: " << temp_dir << std::endl; + std::cout << " Permanent directory: " << permanent_dir << std::endl; + std::cout << std::endl; + std::cout << "Open http://localhost:8080 in your browser to upload files." << std::endl; + + httpserver::webserver ws = httpserver::create_webserver(8080) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(temp_dir) + .generate_random_filename_on_upload() + .file_cleanup_callback([&permanent_dir](const std::string& key, + const std::string& filename, + const httpserver::http::file_info& info) { + (void)key; // Unused in this example + // Move the uploaded file to permanent storage + std::string dest = permanent_dir + "/" + filename; + int result = std::rename(info.get_file_system_file_name().c_str(), dest.c_str()); + + if (result == 0) { + std::cout << "Moved: " << filename << " -> " << dest << std::endl; + return false; // Don't delete - we moved it + } else { + std::cerr << "Failed to move " << filename << ", will be deleted" << std::endl; + return true; // Delete the temp file on failure + } + }); + + file_upload_resource fur; + ws.register_resource("/", &fur); + ws.start(true); + + return 0; +} diff --git a/src/http_request.cpp b/src/http_request.cpp index 68ced762..d589262b 100644 --- a/src/http_request.cpp +++ b/src/http_request.cpp @@ -319,10 +319,21 @@ std::ostream &operator<< (std::ostream &os, const http_request &r) { } http_request::~http_request() { - for ( const auto &file_key : get_files() ) { - for ( const auto &files : file_key.second ) { - // C++17 has std::filesystem::remove() - remove(files.second.get_file_system_file_name().c_str()); + for (const auto& file_key : get_files()) { + for (const auto& files : file_key.second) { + bool should_delete = true; + if (file_cleanup_callback != nullptr) { + try { + should_delete = file_cleanup_callback(file_key.first, files.first, files.second); + } catch (...) { + // If callback throws, default to deleting the file + should_delete = true; + } + } + if (should_delete) { + // C++17 has std::filesystem::remove() + remove(files.second.get_file_system_file_name().c_str()); + } } } } diff --git a/src/httpserver/create_webserver.hpp b/src/httpserver/create_webserver.hpp index fca96d91..31faab02 100644 --- a/src/httpserver/create_webserver.hpp +++ b/src/httpserver/create_webserver.hpp @@ -48,6 +48,10 @@ typedef std::function log_access_ptr; typedef std::function log_error_ptr; typedef std::function psk_cred_handler_callback; +namespace http { class file_info; } + +typedef std::function file_cleanup_callback_ptr; + class create_webserver { public: create_webserver() = default; @@ -364,6 +368,11 @@ class create_webserver { return *this; } + create_webserver& file_cleanup_callback(file_cleanup_callback_ptr callback) { + _file_cleanup_callback = callback; + return *this; + } + private: uint16_t _port = DEFAULT_WS_PORT; http::http_utils::start_method_T _start_method = http::http_utils::INTERNAL_SELECT; @@ -409,6 +418,7 @@ class create_webserver { render_ptr _not_found_resource = nullptr; render_ptr _method_not_allowed_resource = nullptr; render_ptr _internal_error_resource = nullptr; + file_cleanup_callback_ptr _file_cleanup_callback = nullptr; friend class webserver; }; diff --git a/src/httpserver/http_request.hpp b/src/httpserver/http_request.hpp index 142251db..b435cf7f 100644 --- a/src/httpserver/http_request.hpp +++ b/src/httpserver/http_request.hpp @@ -44,6 +44,7 @@ #include "httpserver/http_arg_value.hpp" #include "httpserver/http_utils.hpp" #include "httpserver/file_info.hpp" +#include "httpserver/create_webserver.hpp" struct MHD_Connection; @@ -420,6 +421,12 @@ class http_request { // Populate the data cache unescaped_args void populate_args() const; + file_cleanup_callback_ptr file_cleanup_callback = nullptr; + + void set_file_cleanup_callback(file_cleanup_callback_ptr callback) { + file_cleanup_callback = callback; + } + friend class webserver; friend struct details::modded_request; }; diff --git a/src/httpserver/webserver.hpp b/src/httpserver/webserver.hpp index c7b3a226..31ba0958 100644 --- a/src/httpserver/webserver.hpp +++ b/src/httpserver/webserver.hpp @@ -180,6 +180,7 @@ class webserver { const render_ptr not_found_resource; const render_ptr method_not_allowed_resource; const render_ptr internal_error_resource; + const file_cleanup_callback_ptr file_cleanup_callback; std::shared_mutex registered_resources_mutex; std::map registered_resources; std::map registered_resources_str; diff --git a/src/webserver.cpp b/src/webserver.cpp index f0d28c62..cd8b9b50 100644 --- a/src/webserver.cpp +++ b/src/webserver.cpp @@ -165,7 +165,8 @@ webserver::webserver(const create_webserver& params): tcp_nodelay(params._tcp_nodelay), not_found_resource(params._not_found_resource), method_not_allowed_resource(params._method_not_allowed_resource), - internal_error_resource(params._internal_error_resource) { + internal_error_resource(params._internal_error_resource), + file_cleanup_callback(params._file_cleanup_callback) { ignore_sigpipe(); pthread_mutex_init(&mutexwait, nullptr); pthread_cond_init(&mutexcond, nullptr); @@ -631,6 +632,7 @@ std::shared_ptr webserver::internal_error_page(details::modded_re MHD_Result webserver::requests_answer_first_step(MHD_Connection* connection, struct details::modded_request* mr) { mr->dhr.reset(new http_request(connection, unescaper)); + mr->dhr->set_file_cleanup_callback(file_cleanup_callback); if (!mr->has_body) { return MHD_YES; diff --git a/test/integ/file_upload.cpp b/test/integ/file_upload.cpp index 81ab4de8..f474420d 100644 --- a/test/integ/file_upload.cpp +++ b/test/integ/file_upload.cpp @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -660,6 +661,201 @@ LT_BEGIN_AUTO_TEST(file_upload_suite, file_upload_memory_only_excl_content) ws->stop(); LT_END_AUTO_TEST(file_upload_memory_only_excl_content) +// Test that file cleanup callback returning true causes file deletion (default behavior) +LT_BEGIN_AUTO_TEST(file_upload_suite, file_cleanup_callback_returns_true) + string upload_directory = "."; + + // Track callback invocations + std::vector> callback_invocations; + + auto ws = std::make_unique(create_webserver(PORT) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(upload_directory) + .generate_random_filename_on_upload() + .file_cleanup_callback([&callback_invocations]( + const std::string& key, + const std::string& filename, + const httpserver::http::file_info& info) { + callback_invocations.push_back({key, filename, info.get_file_size()}); + return true; // Delete the file + })); + ws->start(false); + LT_CHECK_EQ(ws->is_running(), true); + + print_file_upload_resource resource; + LT_ASSERT_EQ(true, ws->register_resource("upload", &resource)); + + auto res = send_file_to_webserver(false, false); + LT_ASSERT_EQ(res.first, 0); + LT_ASSERT_EQ(res.second, 201); + + ws->stop(); + + // Verify callback was called with correct parameters + LT_CHECK_EQ(callback_invocations.size(), 1); + LT_CHECK_EQ(std::get<0>(callback_invocations[0]), TEST_KEY); + LT_CHECK_EQ(std::get<1>(callback_invocations[0]), TEST_CONTENT_FILENAME); + LT_CHECK_EQ(std::get<2>(callback_invocations[0]), TEST_CONTENT_SIZE); + + // Verify file was deleted (callback returned true) + map> files = resource.get_files(); + LT_CHECK_EQ(files.size(), 1); + map::iterator file = files.begin()->second.begin(); + LT_CHECK_EQ(file_exists(file->second.get_file_system_file_name()), false); +LT_END_AUTO_TEST(file_cleanup_callback_returns_true) + +// Test that file cleanup callback returning false keeps the file +LT_BEGIN_AUTO_TEST(file_upload_suite, file_cleanup_callback_returns_false) + string upload_directory = "."; + string kept_file_path; + + auto ws = std::make_unique(create_webserver(PORT) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(upload_directory) + .generate_random_filename_on_upload() + .file_cleanup_callback([&kept_file_path]( + const std::string& key, + const std::string& filename, + const httpserver::http::file_info& info) { + (void)key; + (void)filename; + kept_file_path = info.get_file_system_file_name(); + return false; // Keep the file + })); + ws->start(false); + LT_CHECK_EQ(ws->is_running(), true); + + print_file_upload_resource resource; + LT_ASSERT_EQ(true, ws->register_resource("upload", &resource)); + + auto res = send_file_to_webserver(false, false); + LT_ASSERT_EQ(res.first, 0); + LT_ASSERT_EQ(res.second, 201); + + ws->stop(); + + // Verify file still exists (callback returned false) + LT_CHECK_EQ(file_exists(kept_file_path), true); + + // Cleanup: manually delete the file + remove(kept_file_path.c_str()); + LT_CHECK_EQ(file_exists(kept_file_path), false); +LT_END_AUTO_TEST(file_cleanup_callback_returns_false) + +// Test selective cleanup: callback can keep some files and delete others +LT_BEGIN_AUTO_TEST(file_upload_suite, file_cleanup_callback_selective) + string upload_directory = "."; + string kept_file_path; + + auto ws = std::make_unique(create_webserver(PORT) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(upload_directory) + .generate_random_filename_on_upload() + .file_cleanup_callback([&kept_file_path]( + const std::string& key, + const std::string& filename, + const httpserver::http::file_info& info) { + (void)filename; + // Keep first file, delete second + if (key == TEST_KEY) { + kept_file_path = info.get_file_system_file_name(); + return false; // Keep + } + return true; // Delete + })); + ws->start(false); + LT_CHECK_EQ(ws->is_running(), true); + + print_file_upload_resource resource; + LT_ASSERT_EQ(true, ws->register_resource("upload", &resource)); + + // Upload two files + auto res = send_file_to_webserver(true, false); + LT_ASSERT_EQ(res.first, 0); + LT_ASSERT_EQ(res.second, 201); + + ws->stop(); + + map> files = resource.get_files(); + LT_CHECK_EQ(files.size(), 2); + + // First file should exist (callback returned false) + LT_CHECK_EQ(file_exists(kept_file_path), true); + + // Second file should be deleted (callback returned true) + auto file_key_2 = files.find(TEST_KEY_2); + LT_ASSERT_EQ(file_key_2 != files.end(), true); + string deleted_file_path = file_key_2->second.begin()->second.get_file_system_file_name(); + LT_CHECK_EQ(file_exists(deleted_file_path), false); + + // Cleanup: manually delete the kept file + remove(kept_file_path.c_str()); +LT_END_AUTO_TEST(file_cleanup_callback_selective) + +// Test that exception in callback defaults to deleting the file +LT_BEGIN_AUTO_TEST(file_upload_suite, file_cleanup_callback_throws) + string upload_directory = "."; + + auto ws = std::make_unique(create_webserver(PORT) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(upload_directory) + .generate_random_filename_on_upload() + .file_cleanup_callback([]( + const std::string& key, + const std::string& filename, + const httpserver::http::file_info& info) -> bool { + (void)key; + (void)filename; + (void)info; + throw std::runtime_error("Test exception"); + })); + ws->start(false); + LT_CHECK_EQ(ws->is_running(), true); + + print_file_upload_resource resource; + LT_ASSERT_EQ(true, ws->register_resource("upload", &resource)); + + auto res = send_file_to_webserver(false, false); + LT_ASSERT_EQ(res.first, 0); + LT_ASSERT_EQ(res.second, 201); + + ws->stop(); + + // Verify file was deleted (exception causes default delete behavior) + map> files = resource.get_files(); + LT_CHECK_EQ(files.size(), 1); + map::iterator file = files.begin()->second.begin(); + LT_CHECK_EQ(file_exists(file->second.get_file_system_file_name()), false); +LT_END_AUTO_TEST(file_cleanup_callback_throws) + +// Test that no callback defaults to deleting files (backward compatibility) +LT_BEGIN_AUTO_TEST(file_upload_suite, file_cleanup_no_callback_deletes) + string upload_directory = "."; + + auto ws = std::make_unique(create_webserver(PORT) + .file_upload_target(httpserver::FILE_UPLOAD_DISK_ONLY) + .file_upload_dir(upload_directory) + .generate_random_filename_on_upload()); + // No file_cleanup_callback set + ws->start(false); + LT_CHECK_EQ(ws->is_running(), true); + + print_file_upload_resource resource; + LT_ASSERT_EQ(true, ws->register_resource("upload", &resource)); + + auto res = send_file_to_webserver(false, false); + LT_ASSERT_EQ(res.first, 0); + LT_ASSERT_EQ(res.second, 201); + + ws->stop(); + + // Verify file was deleted (default behavior) + map> files = resource.get_files(); + LT_CHECK_EQ(files.size(), 1); + map::iterator file = files.begin()->second.begin(); + LT_CHECK_EQ(file_exists(file->second.get_file_system_file_name()), false); +LT_END_AUTO_TEST(file_cleanup_no_callback_deletes) + LT_BEGIN_AUTO_TEST_ENV() AUTORUN_TESTS() LT_END_AUTO_TEST_ENV()