GitLab

When moving or renewing files, they're first removed and then added and they were being added with the DELETE attribute set. The filenames are displayed in red when staged for deletion/removal, so that's how I noticed. Oops.

This will allow fast/easy display of archive contents without actually reading the archive files.

Introduces some new functions:
- list_archive_contents()
- smb_adddfile_withlist()

A new SMB convenience variable ("tail", aliased as "content" for a file).
A new file detail level ("file_detail_content", exposed in JS as FileBase.DETAIL.CONTENTS) which adds a "content" array property to file metadata objects for JS FileBase.get().

Files already added to the new filebases won't have this archive content automatically - I'm looking into that now (likely a new or updated JS script to run).

Going to use these to store SAUCE "AUTHOR" and "GROUP" fields for files.

This won't impact Synchronet as it has a separate signal handling
thread, but we still need to behave properly for processes that
don't.  I'm also saying that ENOMEM does not indicate a disconnection,
though it may be better to pretend it was disconnected...

Hopefully not introducing any bugs in the process.

Hopefully not introducing any bugs in the process.

This involved the removal of the content-transfer-decoding feature of the SMTP mail server since we need single-part MIME-attached file contents to be stored in their original encoded form (e.g. base64-encoded) and not in binary, for the message body text (where NULs aren't allowed, CR/LF is appended, etc).

I accidentally made this change to smbtxt.c in the new_file_base branch and then copied over here, so there's some unrelated innocuous changes (comment header, removal of SMBCALL) that hopefully won't cause a merge conflict later.

I don't actually receive single-part MIME attached files, so depending on others (e.g. Nelgin and Dream Master) to test for me.

Advertise the system's QWK-ID (a.k.a. BBS ID) in exported echomail messages. This will allow correlation of avatars that were imported via SYNCDATA carried via QWKnet (e.g. DOVE-Net) with messages imported from those same BBSes via FTN.

The alternative was to have SYNCDATA Avatar messages include all the AKAs of each BBS (in the body text), but that was looking like a rather complicated solution. This approach (the BBSID kludge) seems a much simpler solution.

I still need to implement the JS side of this solution however (for avatar lookups using the FTN BBSID), but this was the first important step.

A couple of stragglers here, the isalnum() call caught by an MSVC exception.

When forwarding a single-part MIME-encoded HTML email, the preamble (original message header info) and any user comments, need to be HTML-encoded.

Add the commonly-used "Fwd: " prefix to the default message subject, when forwarding.

This required that smb_getplaintext() no longer always-NULLify the message's text_subtype (e.g. "html"). For single-part messages, this element was getting freed and NULLed.

Add/use a new SMBLIB convenience function to add a string header field, but only if non-NULL: smb_hfield_string()

No longer derive the new message header from the original - too many problems. Simplified the forwarding support smblib (multiple SENDER* and RECIPIENT* header fields no longer expected/supported). Technically, the FORWARDED header field really serves no important purpose now.
A comment can be passed into forwardmail() or (if NULL), will be prompted for. There is an issue where the added comment can be obscured by the receiving mail program if the original message is a multi-part MIME message. Not sure the best solution for that. Still a WIP in that regard.

The convenience pointers to_ext and to_net need to be invalidated (nullified) if we are going to allow forwarding from local email boxes to network mail and keep all the original message header fields in tact.

The "Forwarded" field must now be added *before* any new recipient information (local user extension or network address of the forwarding destination).

This is part of the solution to request #83, #108, and #141.

In support of message bases with vote-messages and poll-closures:

smb_msg_type() returns an enum smb_msg_type inferred from the combination
of attribute flags specified (should match msghdr_t.type).

smb_msg_count() calculates the number of message index records of the
specified type(s) (a bit-field) by reading the message base index file. This
value should only used for info/display purposes, so no locking is performed
to return the result as fast as possible.

This allows an accurate number of "posts" (posted *messages*) to be queried
and displayed to a user when appropriate (e.g. instead of
smb.status.total_msgs or just the size of the index file divided by the size
of an index record). In that case, since vote messages and poll closures
aren't displayed as messages to users, exclude those from the count to reduce
confusion. For message bases that don't support voting, this scheme is not
necessary.

Don't strip the "user@" portion of a *NETADDR header field if the specified net-type is NET_NONE.

easily result in a message body being added to a message base with no
accompanying message header: if the header data was 0-length, the hfield_dat
was not actually allocated and the applicable smbmsg_t convenience pointer was
not set accordingly. This had the side effect that if a message were to be
imported with a blank recipient ("To" field), the header was not created at
all (to support mutliple header, single body msgs, i.e. bulkmail).
The other functions that set the convenience pointers do not treat 0-length
header field data special, so this was inconsistent behavior.
Most message types (e.g. networks) don't want 0-length "to" fields, but that
shoudl be enforced somewhere else, not here.

So on Windows, calling unlock() on a file region that you don't have locked,
incurs a delay. Likewise, calling lock() on a file region that already have
locked incurs a delay. These delays were seriously throttling message imports
on Windows (e.g. importing echomail or QWK packets). What should take a few
seconds, was taking minutes with noticeable delays. This code has been
performing redundant locks/unlocks for over 15 years, so I can only conclude
that the performance penalty is a new behavior introduced in recent version(s)
of Windows.

- REPLYTOLIST (a mime-decoded version of RFC822REPLYTO)
- RECIPIENTLIST (a mime-decoded version of RC822TO)
- RFC822CC (a mime-encoded version of SMB_CARBONCOPY)
- RFC822ORG (a mime-encoded version of SMB_ORGANIZATION)
- RFC822SUBJECT (a mime-encoded version of SUBJECT)

The RFC822* hfields are only created when necessary: there was a MIME-encoded
hfield value received (e.g. by the mailsrvr) for the corresponding hfield.

The to_list and replyto_list convenience pointers now point to the MIME-decoded
(plain text) version of these header fields, since that's what everyone
normally wants to see and use.

The MIME-encoded flavors (RFC822*) are stored for relaying via SMTP or POP3
and retaining all data (no normalization or decoding).

A new auxattr bit has been defined: MSG_HFIELDS_UTF8 (happens to be the same
as P_UTF8 - snicker). This bit will be set in msg.hdr.auxattr when one or more
hfield values are in UTF-8 format. When this flag is not set, all hfield values
are assumed to be CP437 for backwards compatibility.
Since we are using a single flag, all header fields have to use the same
encoding (either CP437 or UTF-8). When the hfield values are all plain ASCII,
there's no difference between CP437 and UTF-8 and the MSG_HFIELDS_UTF8 flag
is not expected to be set, though setting it shouldn't hurt. The RFC822* hfield
values should also include US-ASCII text (using MIME-encoding for any 8-bit
charsets).

smb_get_hfield() function prototype change: the 3rd argument changed from
an hfield_t* to an hfield_t**, so that the caller can actually change the
hfield type (in memory) if they wish. Nobody seemed to be passing any non-NULL
3rd argument value, so this changed appeared safe to make.

FIDOCHARSET
with its own smbmsg_t convenience pointer: ftn_charset.
this allows greater control over when/how this header field is exported.
Constify smb_msg_is_utf8().
Note that the "Editor" hfield type should hold any FTN "NOTE" control line
values (where msg editor details/version are normally propagated over FidoNet).

message body is UTF-8 encoded. This works for either messages received via FTN
(with the "CHRS: UTF-8" control paragraph) or MIME-encoded messages (e.g.
Internet e-mail).

This is to be used by things to display messages. If the viewer (e.g. user's
terminal) supports UTF-8, just send through as-is. If not, translation (e.g. to
CP437) or filtering may be needed.

in clear_convenience_ptrs() - otherwise they can be double-freed when the
message (smbmsg_t) is copied with smb_copymsgmem().

to a text/html part of a multi-part MIME encoded message if no text/plain part
exists.
Changed smbmsg_t.charset to smbmsg_t.text_charset.
Added smbmsg_t.text_subtype (e.g. "plain" or "html").
Fixed issue with parsing parsed plain-text when the charset was the last
element of the MIME-part header.

the (never used) SMB_PRIORITY 32-bit hfield (0 = default/highest priority), to
a new fixed message header field (8-bit) following the X-Priority header field
numbering scheme: 1 = highest, 3 = normal, 5 = lowest and 0 = unspecified.

content-type "charset" parameter value and store in the (new) smbmsg_t.charset.
This is a heap-allocated copy of the text from the message body, so it must be
freed in smb_freemsgmem(). When NULL, the charset can be assumed to be
US-ASCII.

pointer - apparently this has never actually been used before.

- to_list
- cc_list
- replyto_list

These are all comma-separated lists of names/addresses, RFC-822 style
(so quotes and angle-brackets and all that jazz preserved/included)

back in 1993, but were never needed/used.

header fields. Eliminate smb_getcontenttype() - now unnecessary.

and frankly could cause all kinds of havok with multiple thread (e.g. nodes)
pushing and popping SMB's concurrently. eek.

Did not clear/set convenience pointers (e.g. msg.subj), so they would point
to the source message hfield_dat buffers which may be freed or garbage.
Apparently nothing that used smb_copymsgmem() used the convenience pointers
(before now), so it hasn't been a problem before now. <shrug>
It became a problem becaue I'm going to use this function in js_put_msg_header
to fix a problem where the underlying smbmsg_t representation is not current
even after MsgBase.put_msg_header() is used.

for file permissions issues when smblib creates *.sch files.