Before now, the archive formats/types (e.g. for creating QWK/REP packets or
temp file download archives) supported by libarchive have been *hard-coded* in
Synchronet to "zip, 7z, tgz", but if you really want to support the creation
of more archive formats using the internal (libarchive) support in Synchronet,
and your system supports it (e.g. confirmed using archive.js), you can add
those types to this list or remove any that are problematic.

This list does not impact the archive types that can be viewed or extracted
using libarchive.

uncrustify nl_split_if_one_liner setting

... using uncrustify mod_paren_on_return config

White-space changes only, exception being the rare insertion of NL before
closing brace (couldn't find the option to disable that behavior).

I excluded some header files (e.g. sbbs.h) since uncrustify seemed to be doing
more harm than good there. I might just end up applying different set of rules
to .h files.

scfgnet.c:1463:25: warning: result of comparison of constant 100000 with expression of type 'uint16_t' (aka 'unsigned short') is always true

For archives with directories, the first call to extract_files_from_archive() from extract_diz() would create sub-directories in the target (temp) directory, but no files within them.

To correctly solve the original problem identified in commit 79a302f4, introduce/use a new 'recurse' argument to extract_files_from_archive() which means to recursively apply the file_list filter (if specified). Always pass 'with_path' argument as false to prevent sub-dir creation.

The JS Archive.extract() method now excepts an additional boolean argument (recurse) following the file list arguments, default is false.

Remove extra whitespace in Archive JSDOC method descriptions to be consistent with other object/class docs.

Previously, extracted files were always overwritten (so that is the "default" for Archive.extract() and mostly what I'm specifying in the C/C++ code by default now), but this caused a problem with DIZ extraction: archives that contained multiple DIZ files (e.g in sub-directories), the last to be extracted would be used. A maximum of 3 DIZs can be extracted, so it would usually be the 3rd DIZ in the archive if there were that many.

Another solution would be to *only* extract DIZ files from the root of the archive and I should look into that as well, but the always-overwrite behavior also seemed to be wrong, so that *also* needed fixing (allow caller to control behavior).

This fixes issue #317, at least for archives where the root DIZ exists *before* any nested DIZ files. I'll have to try and create a purposeful archive to test the other conditions (where the root DIZ would appear *after* the nested DIZ(s)).

This won't impact Synchronet as it has a separate signal handling
thread, but we still need to behave properly for processes that
don't.  I'm also saying that ENOMEM does not indicate a disconnection,
though it may be better to pretend it was disconnected...

Thanks to TRMB for being the guinea pig, I see now that REP packets can't be opened in append mode because we write and then seek back and write some more in msgtoqwk(). Oops.

18 years, 10 months ago I introduced a bug whereby .MSG files in outgoing REP packets were *always* truncated before newly-exported messages were added. Even though the log message would say "Updating /path/to/HUBID.REP" (rather than the usual "Creating ...") it was actually truncating the .MSG file, thus discarding any existing messages that were not previously successfully sent (!). I'm not sure what the problem was I was trying to solve at the time (some "Unix .rep creation bug") - but the change I made at the time was most definitely was not the correct fix. :-(

How I noticed this problem was the HEADERS.DAT Conference Number check I added to qwk_parse_header_list() back in August of 2019. I've been catching/logging those errors here on Vertrauen and collecting *.rep.bad files from occasional QWKnet node-submitted REP packets, but I didn't look into the cause until today: the HEADERS.DAT and VOTING.DAT files were being correctly appended even though the .MSG file was being truncated, so the files would be out-of-sync and this was the root-cause of the crossed-up message bodies/headers seen on DOVE-Net a year or more ago and apparently also the cause of occasionally lost messages from QWKnet (e.g. DOVE-Net) nodes.

To trigger this bug from the node side, you'd have to create a REP packet with one or more message in it and then fail to send it to your hub (e.g. VERT), for any reason. And then when you attempt another pack/call-out, the previously packed messages would be lost and the HEADERS.DAT file would contain stale/out-of-sync information. 

To simplify things, I'm now just using fopen(..., "ab") (append, binary) - fnopen() should not be needed when opening files in the temp_dir. In append mode, no subsequent fseek(..., SEEK_END) should be needed, so don't do that. And use fprintf() for its intended purpose.

Have msgtoqwk() return a negative number upon error (e.g. blank message, failed to read to body text), which means *no* message header was written to the messages.dat file, so don't increment the record (block) counter used to create the index (NDX) file.

If msgtoqwk() returns 0, that means that a header was written, but no body text (e.g. a vote message) and that is different than an error.

together to keep things atomic:
1. Eliminate eprintf() as sbbs_t::lputs() and sbbs_t::lprintf() do the
   "right thing" (e.g. log to the event log when called from the event_thread).
2. Moved upload_stats() and download_stats() from ftpsrvr.c to getstats.c where
    I plan to make use of them for JS-based upload/download file support.

characters in message headers and body text will be converted to CP437.
Also include a new field in HEADERS.DAT: utf8 = true/false to indicate that
the message headers and body text contain UTF-8 encoding (not CP437).

since the smb_t already contains a subnum and we were passing a pointer to the
sbbs_t::smb to lower-level called-functions anyway.
Let's not mix-up class-scope with function arguments.

format not a string literal and no format arguments [-Wformat-security]

::lprintf() usage.

  Automatically detect child sub-directories and make them available as virtual
  directories for users to download from - only useful via FTP currently.
* Implemented Sorting options for Message Groups and File Libraries:
  Sub-boards within groups and directories within library can now be
  automatically sorted based on the sysop's chosen sort field (long name, short
  name, or internal code)
  - Required that sub-boards are now stored/saved in msgs.cnf based on groups
  - Required QWKnet hub sub-board reference dynamic renumbering
* Implemented "Template" toggle option for sub-boards and file directories:
  A sub-board or directory marked as "template" will be used for newly created
  (or imported) subs/dirs for that group/lib. Clone Options via SCFG will now
  use the "template" item as the source of the cloned settings, if available.

- Include Kludge Lines     (Yes)
- Include VOTING.DAT File   (Yes)
- Include HEADERS.DAT File   (Yes)
- Extended (QWKE) Packets    (No)
- Exported Ctrl-A Codes    (Strip)

SBBS/QWKnet nodes now have more control over the REP packet files created
by their systems. If the "uplink" (hub) is complaining about receiving
unsupported files (e.g. VOTING.DAT) or kluge lines (e.g. @TZ), just turn them
off.

Also, Ctrl-A codes may now be retained in QWKnetmail. And the @REPLYTO kludge
line enabled for the first time (!).

normal messages along with voting data (polls, ballots, etc.) would always be
imported in this order: msgs, polls, ballots/votes, and then poll-closures.
This could result in a confusing order of messages in the local msg base where
there were messages in reply to a poll before the poll appears and other
oddities. Anyway, this is now resolved by placing a msg "header block" for each
vote-data item in the MESSAGES.DAT file. Since there is no body/text blocks,
it should be ignored under normal circumstances, but these header blocks are
only created if VOTING.DAT is enabled anyway.
And now, the VOTING.DAT contains an extra line (empty .ini section) with the
HEADERS.DAT offset associated with the chronology of the item. The format
is still backwards compatible with the earlier builds that included VOTING.DAT
support.

Also, fixed the vote/poll/closure Message-IDs containing a msg number of 0
(while not technically a problem, it wasn't the intention) with the use of the
new function: get_new_msg_number().

to be able to delete messages that have no hfields, for example, with sub-op
'D' command.
Now support polls with comments (displayed before the poll answers).
Poll results/statistics are only displayed to sub-ops and users who have voted
already. This may be optoinal behavior determined by the pollster in the future.
Sub-ops can now add comments to any post using the 'A' command from the
[O]perator menu - use with caution.

- Users can be restricted from voting with the 'V' restriction
- Sub-boards can be disalbled for voting in SCFG
- VOTING.DAT can be include/excluded from QWK packets via user cfg
  (when a VOTING.DAT is received in a REP, the user cfg flag is auto-set)
- Adds several new text.dat lines (if not present in yours, uses the default)

What's not yet implemented:
- Notification of votes on your posted messages
- Method to view/audit all votes
- Polling
- Any special handling to auto-exclude votes from msg-related JavaScripts

scanposts(), and listsub(). These functions are exposed in the JS 'bbs' object
as scan_subs(), scan_msgs(), and list_msgs().
Wherease previously, all of "your messages" (messages addressed to you)
would be found/displayed if the user answered (Y)es to the "Subjects Only?"
prompt, and only un-read message (to you) would be displayed if they answered
(N)o to the "Subjects Only?", providing long-standing confusion about the
different behavior, now the user is prompted whether to display un-read
messages only (in scansubs/scanallsubs), and in the case of the logon
question to "Search all subs for un-read messages to you?", only un-read
messages are listed/displayed, regardless of how the user answers the
"Subjects Only?" question.
SCAN_UNREAD is only applicable when combined with SCAN_TOYOU.

Added new commands to the msg reading prompt: 'N' (list new messages), 'U'
list un-read messages to-you.

2 new text.dat entries: DisplayUnreadMessagesOnlyQ and NoMessagesFound

I've likely broken something though, so if you see weird behaviour with
extended ASCII, let me know please.

unmoderated message in a sub.

longer (use lprintf(LOG_ERR,...) instead, which does).
Added sbbs_t::hacklog() for convenience.

A lot of strcpy->SAFECOPY and sprintf->safe_snprintf/SAFEPRINTF conversions
and other general code cleanup too.

getlasgmsg() needs a uin32_t* as the second arg.
loadmail() needs an int32_t* as the second argument.
loadposts() needs an int32_t as the first argument.

ptr value, sets last_read msg num along with ptr to actual last message num.

were no errors) - this forces the updated scan pointers to be saved.

loadmail() change which had a trickle-up effect to the higher level functions
that use loadmail().

QWK packing events to spew unintended output strings into the event log.

it would always update the message at index offset 0, thus not correctly
marking the sent message as deleted and sending dupes later. This also led
to mild mail database corruption (duplicate message numbers and index
header offsets).

deleted after QWK-packing.