Skip to content
Snippets Groups Projects
Commit 105d5c49 authored by Rob Swindell's avatar Rob Swindell :speech_balloon:
Browse files

Merge branch 'ddup_synchronet_archive_support' into 'master' 

DD Upload Processor: Now supports Synchronet's built-in archive support to extract archive files

See merge request !304
parents 70a6a7df f228de81
No related branches found
No related tags found
2 merge requests!463MRC mods by Codefenix (2024-10-20),!304DD Upload Processor: Now supports Synchronet's built-in archive support to extract archive files
Hide whitespace changes
Inline Side-by-side
xtrn/dd_upload_processor/ddup.js
+ 108
− 84
View file @ 105d5c49
......@@ -9,18 +9,16 @@
* BBS: Digital Distortion
* BBS address: digdist.bbsindex.com
*
* Date Author Description
* Date Author Version Description
* 2009-12-25-
* 2009-12-28 Eric Oulashin Initial development
* 2009-12-29 Eric Oulashin Version 1.00
* Initial public release
* 2022-06-08 Eric Oulashin Version 1.01
* Made fixes to get the scanner functionality working properly in Linux
* 2022-06-11 Eric Oulashin Version 1.02
* Improved file/dir permissions more: Set file permissions too after extracting
* an archive so that they're all readable.
* 2022-06-11 Eric Oulashin Version 1.03
* Removed the chmod stuff, as it is actually not needed.
* 2009-12-28 Eric Oulashin Initial development
* 2009-12-29 Eric Oulashin 1.00 Initial public release
* 2022-06-08 Eric Oulashin 1.01 Made fixes to get the scanner functionality working properly in Linux
* 2022-06-11 Eric Oulashin 1.02 Improved file/dir permissions more: Set file permissions after extracting
* an archive so that they're all readable.
* 2022-06-11 Eric Oulashin 1.03 Removed the chmod stuff, as it is actually not needed.
* 2023-08-06 Eric Oulashin 1.04 Now uses Synchronet's built-in archiver (added in Synchronet 3.19),
* if available, to extract archives.
*/
/* Command-line arguments:
......@@ -48,8 +46,8 @@ gStartupPath = backslash(gStartupPath.replace(/[\/\\][^\/\\]*$/,''));
load(gStartupPath + "ddup_cleanup.js");
// Version information
var gDDUPVersion = "1.03";
var gDDUPVerDate = "2022-06-11";
var gDDUPVersion = "1.04";
var gDDUPVerDate = "2023-08-06";
// Store whether or not this is running in Windows
var gRunningInWindows = /^WIN/.test(system.platform.toUpperCase());
......@@ -822,78 +820,104 @@ function fixPathSlashes(pPath)
// Return value: A blank string on success, or an error message on failure.
function extractFileToDir(pFilename, pWorkDir)
{
// If pFilename doesn't exist, then return with an error.
if (typeof(pFilename) != "string")
return ("Invalid filename specified.");
if (pFilename.length == 0)
return ("No filename specified.");
if (!file_exists(pFilename))
return ("The specified file does not exist.");
// If pWorkDir is blank, then return with an error.
if (typeof(pWorkDir) != "string")
return ("Unknown argument specified for the work directory.");
if (pWorkDir.length == 0)
return ("No work directory specified.");
// If pWorkDir ends with a slash, remove it.
if ((/\/$/.test(pWorkDir)) || (/\\$/.test(pWorkDir)))
pWorkDir = pWorkDir.substr(0, pWorkDir.length-1);
// If the work directory doesn't exist, then return with
// an error.
// Note: file_exists() doesn't seem to work properly with directories.
//if (!file_exists(pWorkDir))
// return ("The work directory doesn't exist.");
var filenameExt = getFilenameExtension(pFilename);
// Return with errors if there are problems.
if (filenameExt.length == 0)
return ("Can't extract (no file extension).");
if (typeof(gFileTypeCfg[filenameExt]) == "undefined")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
if (gFileTypeCfg[filenameExt].extractCmd == "")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
var retval = "";
// Extract the file to the work directory.
var extractCmd = gFileTypeCfg[filenameExt].extractCmd.replace("%FILENAME%", "\"" + fixPathSlashes(pFilename) + "\"");
extractCmd = extractCmd.replace("%FILESPEC% ", "");
extractCmd = extractCmd.replace("%TO_DIR%", "\"" + fixPathSlashes(pWorkDir) + "\"");
var retCode = system.exec(extractCmd);
if (retCode != 0)
return ("Extract failed with exit code " + retCode);
// For each file in the work directory:
// If the file has an extract command
// Extract it to a subdir in the temp dir
// Delete the archive
var files = directory(pWorkDir + "/*");
for (var i in files)
{
// If the file has an extract command, then extract it to a
// temp directory in the work directory.
filenameExt = getFilenameExtension(files[i]);
if ((typeof(gFileTypeCfg[filenameExt]) != "undefined") &&
((gFileTypeCfg[filenameExt].extractCmd != "")))
{
// Create the temp directory and extract the file there.
var workDir = pWorkDir + "/" + getFilenameFromPath(files[i] + "_temp");
if (mkdir(workDir))
retval = extractFileToDir(files[i], workDir);
else
retval = "Unable to create a temporary directory.";
// If there was no problem, then delete the archive file. Otherwise,
// stop going through the list of files.
if (retval.length == 0)
file_remove(files[i]);
else
break;
}
}
// If pFilename doesn't exist, then return with an error.
if (typeof(pFilename) != "string")
return ("Invalid filename specified.");
if (pFilename.length == 0)
return ("No filename specified.");
if (!file_exists(pFilename))
return ("The specified file does not exist.");
// If pWorkDir is blank, then return with an error.
if (typeof(pWorkDir) != "string")
return ("Unknown argument specified for the work directory.");
if (pWorkDir.length == 0)
return ("No work directory specified.");
// If pWorkDir ends with a slash, remove it.
if ((/\/$/.test(pWorkDir)) || (/\\$/.test(pWorkDir)))
pWorkDir = pWorkDir.substr(0, pWorkDir.length-1);
// If the work directory doesn't exist, then return with
// an error.
// Note: file_exists() doesn't seem to work properly with directories.
//if (!file_exists(pWorkDir))
// return ("The work directory doesn't exist.");
var filenameExt = getFilenameExtension(pFilename);
// Return with errors if there are problems.
if (filenameExt.length == 0)
return ("Can't extract (no file extension).");
if (typeof(gFileTypeCfg[filenameExt]) == "undefined")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
if (gFileTypeCfg[filenameExt].extractCmd == "")
return ("Can't extract " + getFilenameFromPath(pFilename) + " (I don't know how).");
var retval = "";
// Extract the file to the work directory.
// If the Archive class is available (added in Synchronet 3.19), then
// use it to extract the archive. Otherwise, use the configured external
// archiver command to extract it.
var builtInExtractSucceeded = false;
if (typeof(Archive) === "function")
{
var arcFilenameFixed = fixPathSlashes(pFilename);
var arcFile = new Archive(arcFilenameFixed);
try
{
// Extract with path information (trust the archive that any
// filename characters are okay). If we don't extract with
// path information, then Synchronet may reject some filenames
// due to certain characters in the filename.
var numFilesExtracted = arcFile.extract(pWorkDir, true);
builtInExtractSucceeded = true;
if (user.is_sysop) console.print("\x01n\r\n\x01gExtracted with Synchronet's built-in Archive support\x01n\r\n\r\n"); // Temporary
}
catch (e)
{
// Synchronet's internal archiver was unable to extract it.
log(LOG_ERR, "DD Upload Processor: Synchronet internal archiver failed to extract " + arcFilenameFixed + ": " + e);
}
}
if (!builtInExtractSucceeded)
{
var extractCmd = gFileTypeCfg[filenameExt].extractCmd.replace("%FILENAME%", "\"" + fixPathSlashes(pFilename) + "\"");
extractCmd = extractCmd.replace("%FILESPEC% ", "");
extractCmd = extractCmd.replace("%TO_DIR%", "\"" + fixPathSlashes(pWorkDir) + "\"");
var retCode = system.exec(extractCmd);
if (retCode != 0)
return ("Extract failed with exit code " + retCode);
}
// For each file in the work directory:
// If the file has an extract command
// Extract it to a subdir in the temp dir
// Delete the archive
var files = directory(pWorkDir + "/*");
for (var i in files)
{
// If the file has an extract command, then extract it to a
// temp directory in the work directory.
filenameExt = getFilenameExtension(files[i]);
if ((typeof(gFileTypeCfg[filenameExt]) != "undefined") && ((gFileTypeCfg[filenameExt].extractCmd != "")))
{
// Create the temp directory and extract the file there.
var workDir = pWorkDir + "/" + getFilenameFromPath(files[i] + "_temp");
if (mkdir(workDir))
retval = extractFileToDir(files[i], workDir);
else
retval = "Unable to create a temporary directory.";
return retval;
// If there was no problem, then delete the archive file. Otherwise,
// stop going through the list of files.
if (retval.length == 0)
file_remove(files[i]);
else
break;
}
}
return retval;
}
// This function executes an OS command and returns its output as an
......
xtrn/dd_upload_processor/readme.txt
+ 22
− 10
View file @ 105d5c49
Digital Distortion Upload Processor
Version 1.03
Release date: 2022-06-11
Version 1.04
Release date: 2023-08-06
by
Eric Oulashin
Sysop of Digital Distortion BBS
BBS internet address: digdist.bbsindex.com
BBS internet address: digdist.synchro.net
AKA digitaldistortionbbs.com
Email: eric.oulashin@gmail.com
......@@ -42,9 +43,12 @@ to scan uploaded files, with the added ability to extract compressed files
in order to scan the files inside the compressed file.
File formats can be specified and configured via a configuration file,
including extraction commands for compressed files. In addition, the
virus scan command can be configured in the main configuration file,
which should allow for the use of any virus scanner, as long as it is a
including extraction commands for compressed files. Synchronet's built-in
archive support (added in Synchronet 3.19) will be used, if available, to
extract archive files. Otherwise, extract commands for archive file types can
be configured in the archive file type configuration file (ddup_file_types.cfg).
In addition, the virus scan command can be configured in the main configuration
file, which should allow for the use of any virus scanner, as long as it is a
command-line scanner (no GUI) and is able to take a subdirectory as a
command-line parameter.
......@@ -76,10 +80,18 @@ uploaded file to be rejected.
3. Archive File Software
========================
Digital Distortion Upload Processor comes with configuration settings to
use various archivers to handle extraction of ZIP, 7Z (7-Zip), RAR, ARJ, MSI,
TAR, GZ, TGZ, and TAR.GZ archives. If you want to use other archiver programs,
you will need to update the configuration to modify the commands used.
Digital Distortion Upload Processor will (try to) use Synchronet's built-in
archive support (if available) to extract archive files. If that fails, it will
fall back on a configured command-line to extract the archive (if configured).
Synchronet's built-in archive support will take precedence over any configured
extraction command (that is, if Synchronet itself is able to extract the
archive, any configured extraction command for that archive file type will not
be uesd).
Configuration settings to use various archivers to handle extraction of ZIP, 7Z
(7-Zip), RAR, ARJ, MSI, TAR, GZ, TGZ, and TAR.GZ archives. If you want to use
other archiver programs, you will need to update the configuration to modify the
commands used.
The file format configuration file included with this script includes
extraction command lines (specified by an EXTRACT setting) for various
......
xtrn/dd_upload_processor/version_history.txt
+ 2
− 0
View file @ 105d5c49
......@@ -2,6 +2,8 @@ Revision History for Digital Distortion Upload Processor
========================================================
Version Date Description
------- ---- -----------
1.04 2023-08-06 Now uses Synchronet's built-in archiver (added in
Synchronet 3.19), if available, to extract archives.
1.03 2022-06-11 Removed the chmod updates because they're actually not
needed. Some antivirus scanners (such as ClamAV) need
a command line parameter to pass the file permissions
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment