Skip to content
Snippets Groups Projects
Commit 105d5c49 authored by Rob Swindell's avatar Rob Swindell :speech_balloon:
Browse files

Merge branch 'ddup_synchronet_archive_support' into 'master'

DD Upload Processor: Now supports Synchronet's built-in archive support to extract archive files

See merge request !304
parents 70a6a7df f228de81
No related branches found
No related tags found
2 merge requests!463MRC mods by Codefenix (2024-10-20),!304DD Upload Processor: Now supports Synchronet's built-in archive support to extract archive files
...@@ -9,18 +9,16 @@ ...@@ -9,18 +9,16 @@
* BBS: Digital Distortion * BBS: Digital Distortion
* BBS address: digdist.bbsindex.com * BBS address: digdist.bbsindex.com
* *
* Date Author Description * Date Author Version Description
* 2009-12-25- * 2009-12-25-
* 2009-12-28 Eric Oulashin Initial development * 2009-12-28 Eric Oulashin Initial development
* 2009-12-29 Eric Oulashin Version 1.00 * 2009-12-29 Eric Oulashin 1.00 Initial public release
* Initial public release * 2022-06-08 Eric Oulashin 1.01 Made fixes to get the scanner functionality working properly in Linux
* 2022-06-08 Eric Oulashin Version 1.01 * 2022-06-11 Eric Oulashin 1.02 Improved file/dir permissions more: Set file permissions after extracting
* Made fixes to get the scanner functionality working properly in Linux * an archive so that they're all readable.
* 2022-06-11 Eric Oulashin Version 1.02 * 2022-06-11 Eric Oulashin 1.03 Removed the chmod stuff, as it is actually not needed.
* Improved file/dir permissions more: Set file permissions too after extracting * 2023-08-06 Eric Oulashin 1.04 Now uses Synchronet's built-in archiver (added in Synchronet 3.19),
* an archive so that they're all readable. * if available, to extract archives.
* 2022-06-11 Eric Oulashin Version 1.03
* Removed the chmod stuff, as it is actually not needed.
*/ */
/* Command-line arguments: /* Command-line arguments:
...@@ -48,8 +46,8 @@ gStartupPath = backslash(gStartupPath.replace(/[\/\\][^\/\\]*$/,'')); ...@@ -48,8 +46,8 @@ gStartupPath = backslash(gStartupPath.replace(/[\/\\][^\/\\]*$/,''));
load(gStartupPath + "ddup_cleanup.js"); load(gStartupPath + "ddup_cleanup.js");
// Version information // Version information
var gDDUPVersion = "1.03"; var gDDUPVersion = "1.04";
var gDDUPVerDate = "2022-06-11"; var gDDUPVerDate = "2023-08-06";
// Store whether or not this is running in Windows // Store whether or not this is running in Windows
var gRunningInWindows = /^WIN/.test(system.platform.toUpperCase()); var gRunningInWindows = /^WIN/.test(system.platform.toUpperCase());
...@@ -822,78 +820,104 @@ function fixPathSlashes(pPath) ...@@ -822,78 +820,104 @@ function fixPathSlashes(pPath)
// Return value: A blank string on success, or an error message on failure. // Return value: A blank string on success, or an error message on failure.
function extractFileToDir(pFilename, pWorkDir) function extractFileToDir(pFilename, pWorkDir)
{ {
// If pFilename doesn't exist, then return with an error. // If pFilename doesn't exist, then return with an error.
if (typeof(pFilename) != "string") if (typeof(pFilename) != "string")
return ("Invalid filename specified."); return ("Invalid filename specified.");
if (pFilename.length == 0) if (pFilename.length == 0)
return ("No filename specified."); return ("No filename specified.");
if (!file_exists(pFilename)) if (!file_exists(pFilename))
return ("The specified file does not exist."); return ("The specified file does not exist.");
// If pWorkDir is blank, then return with an error. // If pWorkDir is blank, then return with an error.
if (typeof(pWorkDir) != "string") if (typeof(pWorkDir) != "string")
return ("Unknown argument specified for the work directory."); return ("Unknown argument specified for the work directory.");
if (pWorkDir.length == 0) if (pWorkDir.length == 0)
return ("No work directory specified."); return ("No work directory specified.");
// If pWorkDir ends with a slash, remove it. // If pWorkDir ends with a slash, remove it.
if ((/\/$/.test(pWorkDir)) || (/\\$/.test(pWorkDir))) if ((/\/$/.test(pWorkDir)) || (/\\$/.test(pWorkDir)))
pWorkDir = pWorkDir.substr(0, pWorkDir.length-1); pWorkDir = pWorkDir.substr(0, pWorkDir.length-1);
// If the work directory doesn't exist, then return with // If the work directory doesn't exist, then return with
// an error. // an error.
// Note: file_exists() doesn't seem to work properly with directories. // Note: file_exists() doesn't seem to work properly with directories.
//if (!file_exists(pWorkDir)) //if (!file_exists(pWorkDir))
// return ("The work directory doesn't exist."); // return ("The work directory doesn't exist.");
var filenameExt = getFilenameExtension(pFilename); var filenameExt = getFilenameExtension(pFilename);
// Return with errors if there are problems. // Return with errors if there are problems.
if (filenameExt.length == 0) if (filenameExt.length == 0)
return ("Can't extract (no file extension)."); return ("Can't extract (no file extension).");
if (typeof(gFileTypeCfg[filenameExt]) == "undefined") if (typeof(gFileTypeCfg[filenameExt]) == "undefined")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how)."); return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
if (gFileTypeCfg[filenameExt].extractCmd == "") if (gFileTypeCfg[filenameExt].extractCmd == "")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how)."); return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
var retval = ""; var retval = "";
// Extract the file to the work directory. // Extract the file to the work directory.
var extractCmd = gFileTypeCfg[filenameExt].extractCmd.replace("%FILENAME%", "\"" + fixPathSlashes(pFilename) + "\""); // If the Archive class is available (added in Synchronet 3.19), then
extractCmd = extractCmd.replace("%FILESPEC% ", ""); // use it to extract the archive. Otherwise, use the configured external
extractCmd = extractCmd.replace("%TO_DIR%", "\"" + fixPathSlashes(pWorkDir) + "\""); // archiver command to extract it.
var retCode = system.exec(extractCmd); var builtInExtractSucceeded = false;
if (retCode != 0) if (typeof(Archive) === "function")
return ("Extract failed with exit code " + retCode); {
// For each file in the work directory: var arcFilenameFixed = fixPathSlashes(pFilename);
// If the file has an extract command var arcFile = new Archive(arcFilenameFixed);
// Extract it to a subdir in the temp dir try
// Delete the archive {
var files = directory(pWorkDir + "/*"); // Extract with path information (trust the archive that any
for (var i in files) // filename characters are okay). If we don't extract with
{ // path information, then Synchronet may reject some filenames
// If the file has an extract command, then extract it to a // due to certain characters in the filename.
// temp directory in the work directory. var numFilesExtracted = arcFile.extract(pWorkDir, true);
filenameExt = getFilenameExtension(files[i]); builtInExtractSucceeded = true;
if ((typeof(gFileTypeCfg[filenameExt]) != "undefined") && if (user.is_sysop) console.print("\x01n\r\n\x01gExtracted with Synchronet's built-in Archive support\x01n\r\n\r\n"); // Temporary
((gFileTypeCfg[filenameExt].extractCmd != ""))) }
{ catch (e)
// Create the temp directory and extract the file there. {
var workDir = pWorkDir + "/" + getFilenameFromPath(files[i] + "_temp"); // Synchronet's internal archiver was unable to extract it.
if (mkdir(workDir)) log(LOG_ERR, "DD Upload Processor: Synchronet internal archiver failed to extract " + arcFilenameFixed + ": " + e);
retval = extractFileToDir(files[i], workDir); }
else }
retval = "Unable to create a temporary directory."; if (!builtInExtractSucceeded)
{
// If there was no problem, then delete the archive file. Otherwise, var extractCmd = gFileTypeCfg[filenameExt].extractCmd.replace("%FILENAME%", "\"" + fixPathSlashes(pFilename) + "\"");
// stop going through the list of files. extractCmd = extractCmd.replace("%FILESPEC% ", "");
if (retval.length == 0) extractCmd = extractCmd.replace("%TO_DIR%", "\"" + fixPathSlashes(pWorkDir) + "\"");
file_remove(files[i]); var retCode = system.exec(extractCmd);
else if (retCode != 0)
break; return ("Extract failed with exit code " + retCode);
} }
} // For each file in the work directory:
// If the file has an extract command
// Extract it to a subdir in the temp dir
// Delete the archive
var files = directory(pWorkDir + "/*");
for (var i in files)
{
// If the file has an extract command, then extract it to a
// temp directory in the work directory.
filenameExt = getFilenameExtension(files[i]);
if ((typeof(gFileTypeCfg[filenameExt]) != "undefined") && ((gFileTypeCfg[filenameExt].extractCmd != "")))
{
// Create the temp directory and extract the file there.
var workDir = pWorkDir + "/" + getFilenameFromPath(files[i] + "_temp");
if (mkdir(workDir))
retval = extractFileToDir(files[i], workDir);
else
retval = "Unable to create a temporary directory.";
return retval; // If there was no problem, then delete the archive file. Otherwise,
// stop going through the list of files.
if (retval.length == 0)
file_remove(files[i]);
else
break;
}
}
return retval;
} }
// This function executes an OS command and returns its output as an // This function executes an OS command and returns its output as an
......
Digital Distortion Upload Processor Digital Distortion Upload Processor
Version 1.03 Version 1.04
Release date: 2022-06-11 Release date: 2023-08-06
by by
Eric Oulashin Eric Oulashin
Sysop of Digital Distortion BBS Sysop of Digital Distortion BBS
BBS internet address: digdist.bbsindex.com BBS internet address: digdist.synchro.net
AKA digitaldistortionbbs.com
Email: eric.oulashin@gmail.com Email: eric.oulashin@gmail.com
...@@ -42,9 +43,12 @@ to scan uploaded files, with the added ability to extract compressed files ...@@ -42,9 +43,12 @@ to scan uploaded files, with the added ability to extract compressed files
in order to scan the files inside the compressed file. in order to scan the files inside the compressed file.
File formats can be specified and configured via a configuration file, File formats can be specified and configured via a configuration file,
including extraction commands for compressed files. In addition, the including extraction commands for compressed files. Synchronet's built-in
virus scan command can be configured in the main configuration file, archive support (added in Synchronet 3.19) will be used, if available, to
which should allow for the use of any virus scanner, as long as it is a extract archive files. Otherwise, extract commands for archive file types can
be configured in the archive file type configuration file (ddup_file_types.cfg).
In addition, the virus scan command can be configured in the main configuration
file, which should allow for the use of any virus scanner, as long as it is a
command-line scanner (no GUI) and is able to take a subdirectory as a command-line scanner (no GUI) and is able to take a subdirectory as a
command-line parameter. command-line parameter.
...@@ -76,10 +80,18 @@ uploaded file to be rejected. ...@@ -76,10 +80,18 @@ uploaded file to be rejected.
3. Archive File Software 3. Archive File Software
======================== ========================
Digital Distortion Upload Processor comes with configuration settings to Digital Distortion Upload Processor will (try to) use Synchronet's built-in
use various archivers to handle extraction of ZIP, 7Z (7-Zip), RAR, ARJ, MSI, archive support (if available) to extract archive files. If that fails, it will
TAR, GZ, TGZ, and TAR.GZ archives. If you want to use other archiver programs, fall back on a configured command-line to extract the archive (if configured).
you will need to update the configuration to modify the commands used. Synchronet's built-in archive support will take precedence over any configured
extraction command (that is, if Synchronet itself is able to extract the
archive, any configured extraction command for that archive file type will not
be uesd).
Configuration settings to use various archivers to handle extraction of ZIP, 7Z
(7-Zip), RAR, ARJ, MSI, TAR, GZ, TGZ, and TAR.GZ archives. If you want to use
other archiver programs, you will need to update the configuration to modify the
commands used.
The file format configuration file included with this script includes The file format configuration file included with this script includes
extraction command lines (specified by an EXTRACT setting) for various extraction command lines (specified by an EXTRACT setting) for various
......
...@@ -2,6 +2,8 @@ Revision History for Digital Distortion Upload Processor ...@@ -2,6 +2,8 @@ Revision History for Digital Distortion Upload Processor
======================================================== ========================================================
Version Date Description Version Date Description
------- ---- ----------- ------- ---- -----------
1.04 2023-08-06 Now uses Synchronet's built-in archiver (added in
Synchronet 3.19), if available, to extract archives.
1.03 2022-06-11 Removed the chmod updates because they're actually not 1.03 2022-06-11 Removed the chmod updates because they're actually not
needed. Some antivirus scanners (such as ClamAV) need needed. Some antivirus scanners (such as ClamAV) need
a command line parameter to pass the file permissions a command line parameter to pass the file permissions
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment