Before now, the archive formats/types (e.g. for creating QWK/REP packets or
temp file download archives) supported by libarchive have been *hard-coded* in
Synchronet to "zip, 7z, tgz", but if you really want to support the creation
of more archive formats using the internal (libarchive) support in Synchronet,
and your system supports it (e.g. confirmed using archive.js), you can add
those types to this list or remove any that are problematic.

This list does not impact the archive types that can be viewed or extracted
using libarchive.

No change in functionality, just code comprehension

... using uncrustify mod_paren_on_return config

White-space changes only, exception being the rare insertion of NL before
closing brace (couldn't find the option to disable that behavior).

I excluded some header files (e.g. sbbs.h) since uncrustify seemed to be doing
more harm than good there. I might just end up applying different set of rules
to .h files.

e.g. [can|is]_subject_something() is now named subject_[can|is]_something()

No functional change.

These @-codes can be used to construct ftp[s] and http[s] URLs to display to
users. For use in new/optional display file text/menu/download.*

Add optional configurable 'vpath' (per-directory) for directories that have
web/ftp aliases, so they preferred/short path (alias) will be used in the
expanded @-codes.

In SCFG, display each directory's virtual file path ([auto-generated] or set
manually).

getfilevpath() no longer assumes the target buf is >= MAX_PATH+1 bytes long

Add dir_vpath() to get a directory's vpath

Extend maximum file library parent directory from 47 to 100 chars. This limit
was likely imposed because we didn't have horiztonal scrolling input in UIFC
getstr() support at the time. We're no longer limited by that.

Returns count of files removed now (not bool).

Commit 3a3c889b (2 years ago now) changed loadfiles() to use liberal file
matching (e.g. "syncterm.exe" matched both "syncterm.exe" and
"syncterm_v1.2b.exe").

This could produce surprising results when doing file list querieis/operations
with the FileBase methods via JS (e.g. jsexec utils) and (now that I look at
it), the FTP server too.

So we should not have been doing liberal file matching *everywhere* loadfiles
is used, just where it was a usability issue (due to displayed filenames being
truncated to 12 chars for <=80 column terminals).

Now solved by add/use of new liberal_filepattern() function only in the
built-in file listing methods: sbbs_t::listfiles() and sbbs_t::listfileinfo().

Note: Custom JS file searching/listing scripts may now need their own
work-arounds for this usability issue, if they have it.

This fixes a filebase corruption issue that could be triggered by using
FileBase.update() to rename a file while also changing the file's auxdata or
extended description: you can't remove a file header that has the new filename
(in the file_t.name field), cause it doesn't exist yet. So we needed an SMBLIB
API function to pass the (original) filename instead. Much like the filedat.c
removefile() function, but without the SMB opening/closing feature.

Fixes cosmetic/readability issue when imported DIZ files that have line/word
wrapped text and the space between words was being removed.

client_t.protocol is no longer a pointer.

Warnings reported by gcc (Debian 12.2.0-14) 12.2.0

The fallback to external extractors/archivers didn't really work unless a
libarchive function (called from extract_files_from_archive) actually failed
and the return value was < 0. A return value of 0 would mean the external
file extractor would never be used.

This was discovered when trying to import DIZ from old ZIP files (from 1992)
that used "Implode" compression method for the FILE_ID.DIZ, which is a
compression method *not* supported by libarchive. This is a reason why sysops
might want to leave Info-zip's 'unzip' as a configured Extractable File Type
handler (in SCFG) for 'zip' files.

Don't call the external archiver 3 times for the 3 DIZ filenames supported.
Just call the external archiver once, and pass all 3 filesnames. This also
means that the call to system() can return non-zero (e.g. Info-zip 'unzip'
will return 2 if any of the files aren't present in the archive, even if
one is) - so ignore this return value from system(). This speeds up bulk
import (e.g. using addfiles.js).

Unfortunately, I couldn't really find a nice cross-platform way to suppress
the unzip "caution: filename not matched" message sent to stderr. That's
a bummer and a little annoying.

Ignore 0-length DIZ files.

That's percent minus, to go with percent plus (user's real name).

The string is auto-quoted if it contains a space.

For Keyop's use with rlogin.js, maybe.

No obvious bug fixes here, just future proofing.

So Clang-FreeBSD was warning (in compiles of scfg/scfg*.c by Deuce):
result of comparison of constant 100000 with expression of type 'uint16_t'
(aka 'unsigned short') is always true

Why? Cause a uint16_t's max value is 65535 (less than 100000). Sure we could
have just lowered the UIFC max number of config items to 65535, but that would
have been too easy. And why are these compared-with values of type uint16_t to
begin with? Because most ctrl/*.cnf lists (of configuration items) were
limited to 65535 entries cause ... 16-bit DOS, historically. Now that *.cnf
files aren't used, we could just increase these scfg_t.*_total type sizes from
16 to 32-bits, yeah? The result is this commit.

I went to (signed) int so we could still keep -1 as the special illegal
sub/dir num value (e.g. INVALID_SUB, which is sometimes used to indicate the
email message base). Theoretically, 2 billion configuration items could be
supported in these lists, but SCFG will limit you to 100000 anyway. So there's
a whole lot of s/uint/int in this commit.

I'd be very surprised if this doesn't result in some new GCC/Clang warnings,
but at least the old "comparison of constant 100000" warnings are now gone!

A bunch of possible (but often, not really) use of undefined values.
Some ignored return values (e.g. of chsize/ftruncate, read, write, fgets).

Other than some added diagnostics upon some of these unexpected syscall
failures, there should be no change in behavior from this commit.

For archives with directories, the first call to extract_files_from_archive() from extract_diz() would create sub-directories in the target (temp) directory, but no files within them.

To correctly solve the original problem identified in commit 79a302f4, introduce/use a new 'recurse' argument to extract_files_from_archive() which means to recursively apply the file_list filter (if specified). Always pass 'with_path' argument as false to prevent sub-dir creation.

The JS Archive.extract() method now excepts an additional boolean argument (recurse) following the file list arguments, default is false.

Remove extra whitespace in Archive JSDOC method descriptions to be consistent with other object/class docs.

Handles integer overflow in the summing of user's credits and remaining daily
free credits.

Default to 100000 (cps) if passed a rate_cps argument value of 0.

This is just extra paranoia now since the reading of file.ini enforces the value range
8 <-> 65535 now.

Allows for more/better wildcard matching in viewable and testable
file types and download-events (e.g. "tar.gz" or "tar.*").

This solves the "double-dot" problem in some file types/extensions.
An implicit "*." is prepended before the configured file type/extension.

Now, if the BOOL argument value is FALSE, then the .ini file is opened read-only (and thus only read permissions are needed, resolving issue #455).

If the BOOL argument value is TRUE, create-if-not-exist is implied. There are no use cases where we would want to open an .ini file for writing only if it already existed.

Address issue reported by Nightfox via DOVE-Net:
Today I was using the Archive class to extract exactly one file from a zip
file, and I'm a little confused on the exception throwing behavior regarding
the max_files parameter.  When calling extract(), I gave it a filename pattern
and expected exactly 1 file to be extracted, so I also gave a max_files
argument as 1.  It extracted the one file, but it threw an exception with the
error "Error: maximum number of files (1) extracted (after extracting 1 item
successfully)".

Should that be an error condition to throw an exception? I expected 1 file to
be extracted, and that file was extracted successfully.  If I specify max_files
as 2, then it doesn't throw an exception.   

1. Was not setting f->dir to the correct directory number, so only ftp-uploads to the *first* directory (dirnum = 0) would extract DIZ files of uploaded files.

Removing the 'dirnum' parameter to addfile() since that implied that you did not have to initialize the 'dir' element of the passed file_t, but you do: to get the correct file path for file size/date detection and the DIZ extraction.

2. Was getting heap-corruption when freeing the imported/formatted DIZ text on Windows once the above problem was fixed: can't free() in one DLL memory that was allocated in another DLL. Created and now using free_diz() to free the memory allocated in read_diz().

format_diz() handles a NULL 'lines' argument correctly/gracefully, so no need for the NULL lines check in sbbs_t::uploadfile().

Added FTP server log messages for successful file upload or update by user.

When loadfiles() calls sortfiles(), only the file's index records have been
read in, so trying to sort on any header field won't work.

This bug wasn't observable when sorting by date ascending, since that's the
natural index order of the files already (order imported/added), only
observed when sorting by date descending (newest at the top).

(I'm looking at you, mist1221.zip)

... so first try to extract DIZ files from the root of the archive, then try again searching nested directories too. <sigh>

Credits and daily free credits are accurate to the byte up to (a maximum) of 18446744073709551615 (that's 18 Exbibytes - 1).

User's upload and download byte stats are now similarly extended in maximum range, but the accuracy is only "to the byte" for values less than 10,000,000,000. Beyond that value, the accuracy declines, but is generally pretty damn accurate (to 4 decimal places beyond the nearest multiple of a power of 1024), so I don't expect that to be an issue. This method of storing upload/download byte stats allowed me to use the same 10-character user record fields in the user.dat file.

As a side-effect of this enhancements:
* User and file credit values are now expressed in multiples of powers of 1024 (e.g. 4.0G rather than 4,294,967,296).
* Free credits per day per security level has now been extended from 32 to 64-bits (to accommodate values >= 4GB).
* adjustuserrec() now longer takes the record length since we can easily determine that automatically and don't need more "sources of truth" that can be out-of-sync (e.g. the U_CDT field length going from 10 to 20 chars with this change).
* setting the stage for locale-dependent thousands-separators (e.g. space instead of comma) - currently still hard-coded to comma
* more/better support for files > 4GB in size (e.g. in the batch download queue)
* user_t ulong fields changed to either uint32_t or uint64_t - I didn't realize how many long/ulong's remained in the code (which are sometmies 32-bit, sometimes 64-bit) - ugh
* Steve's ultoac() function renamed to u32toac() and created a C++ wrapper that still uses the old name, for homage

Used the reserved 16-bits in the file index record to extend the supported index-file-size from 4294967295 (4GB) to 281474976710655 (281TB). I think that's big enough for the foreseeable future. :-)

Size is indexed, so might as well sort by it. This does have an issue with files >= 4GB in length however, so I'm looking at that next.

If the extended description is UTF-8, first convert it to CP437.

This change is just for internal consistency and convenience right now: the lib_t.vdir is a "sanitized" copy of the lib's short name (spaces are converted to dots or underscores based on the logic that the FTP server used in dotname()) and the dir_t.vdir is just a pointer to the dir's code_suffix. No other permutations are made (e.g. lower-casing the strings). Although the virtual directory names of libraries will now appear in mixed case in the FTP server (previously, they were all lowercase), the directory names are actually treated case-insensitively, so it should not make any difference. If forced-lowercase is preferred for some reason, please speak up.

This change leads the way to eventually, possibly, making these virtual path elements sysop-configurable. For now, it's just better to have a *copy* of the lib's short name that is appropriately modified to make a suitable directory name and have that vpath element available globally (to all servers and services) in a consistent manner.

So Nelgin asked (about filebase access via http), what if the library short name has a space in it? The answer now is, the spaces are replaced with a '.' or '_' (if there's already dots in the name).

This fixes issue #328.

The user actually *can* remove files from the batch queues in v3.19b, but you have to type the filenames which is not obvious from the prompt which implies you need to type the file index position (e.g. '1' for the first file in the queue). In all Synchronet versions prior, you could only remove by number (and not by name).

The fix is to allow either the number or the name of the file to be entered at the RemoveWhich prompt and the file is removed from the queue successfully.

Thanks Ragnarok!

The file_list[] parameter was expected to contain only files, but the directory() function (used to create that file_list[]) returns a list of all directory entries, including sub-directories. I could (and maybe will) add an option to directory() to only include files or dirs, but this seemed the more direct fix for the problem reported by DesotoFireflite (VALHALLA):

TickIT's nodelist_handler.js appears to be creating and leaving behind a sub-directory of the temp directory, triggering this error:
 1/23  11:36:56a  QNET libarchive error -1 (13 opening c:\SBBS\temp\event\nodelist_handler/) creating c:\SBBS\data\VERT.REP 

Why isn't the temp directory fully cleaned up after/between events? That's another thing to look into.

file_area.web_vpath_prefix
file-metadata-object (return value of FileBase.get()).vpath

CID 345291
It's actually a false positive because if an extension (".suffix") exists in filespec, it must also exist in newfilespec since it's a copy, but whatever. It's better form to check.

Default to 64 characters. Maximum value is 65535 characters, but filenames larger than 64 characters may be problematic (e.g. searching for them, displaying them, security concerns), so only increase with caution. Shorter values are fine, but 0 will just revert back to the default.