e.g. User.downlaoded_file() will now publish to the appropriate MQTT topic and changes to system.node_list[] will get published to MQTT.

Since we're logging at INFO level when the connect is attempted, we can assume
success when we start logging additional init messages.

We need to call mqtt_shutdown() instead of mqtt_close() to have the mosquitto
(loop) thread stopped.

Upon connect failure, call the mqtt_shutdown() *before* calling lprintf->lputs,
which would eventually try to MQTT-publish the log message.

The call to mqtt_connect() can block for a while, so raise the log severity
of the "connecting to broker" message from DEBUG to INFO. Otherwise, a bad
MQTT broker address or port would make the servers just appear to hang during
initialization, for no reason.

Apparently if you call mosquitto_publish_v5() with a non-NULL properties
pointer, it'll only publish via protocol v5 and doesn't work if you're using
protocol v3 connection.

Thanks Ree.

... regardless of the configured MQTT protocol version (v5 or v3.x).

Also include an MQTT v5 user-property ("time") with the message's origination
time/date stamp in ISO-8601 format in all log messages (to both topics).

This will at least double the amount of MQTT log traffic to the broker. When
MQTT v5 is used, the additional user-properties will increase it even a little
more.

Is this what you were suggesting Ree?

The node message should be terminated with a newline character.

e.g.
$ echo "Hey you" | mosquitto_pub -s -t sbbs/VERT/node/1/msg

I'm considering the ability to send user message (telegrams) via MQTT too.

A follow-up to commit 81d4575e

Although I was not able to successfully reproduce the problem that Ree
reported with his commit (even when changing the SCFG->Networks->MQTT->Publish
QOS to 1: At least once) on Windows, I do see how this problem could
theoretically happen. And like Ree said in the follow-up comment on the MR
"maybe these two lines should have stayed in mqtt_startup", they don't really
belong in the connection callback.

The "client" topics only needs to be cleared upon startup or recycle (by
publishing a null message) and it would be bad to clear these topics whenever
the broker was reconnected (the server's clients didn't magically disconnect).
So these "client" topic-clearing publishes are now only done during startup
(again).

The "recycle" topics don't really need to be published to here at all. I
think I only did this for cases where someone published a non-null message to
the topic and its stale message would remain afterward, appearing in MQTT
browsers (like MQTT explorer) long after the server had recycled. The real
solution to this cosmetic issue is to only publish null (0-length) messages to
the "recycle" topics in the first place.

- sbbs/BBSID/exec (publish the timed event's internal code)
- sbbs/BBSID/call (publish the QWKnet hub's ID)

The message contents are not case-sensitive.

e.g. publishing "0" to node/#/set/intr will clear the node-interrupt flag.
Any non-zero message value will "set" the flag. Same is true for the
lock, down, and rerun topics/flags.

These node attributes (misc) flags could be cleared previously
by setting the 'misc' topic (e.g. to 0), but since that's not an atomic
read/modify/write operation, other set misc flags could be lost clearing flags
in that manner.

<nelgin> can I push an intr instruction to a node with mqtt?

node/#/set/status - set the node status value (to an integer)
node/#/set/errors - set the node error counter (to an integer, e.g. "0")
node/#/set/misc - set the node's miscellaneous attributes/flags value
 (hexadecimal values can be set by including "0x" prefix)

The message (payload) doesn't matter for the following topics:
node/#/set/lock - lock a node (no one but sysop can login)
node/#/set/intr - interrupt a node (disconnect a user)
node/#/set/down - down a node (not available for connections)
node/#/set/rerun - rerun a node (reload config upon next connection)

Fixes a stale non-zero error_count retained topic.

Publish all login-failures to this new topic.

This commit also includes a few 64-bit free disk space fixes in the FTP server.

e.g. "action/hack/smtp login" -> "smtp_login", best practices and all.

- Most published messages (besides log entries) have a timestamp (in ISO8601 format) prepended and tab-separated
- The order and number of elements in client messages (list and activities) has been updated, now includes user number
- Server client lists are now published to .../SERVER/client/list
- Server client activities (connect, disconnect, update) are now published to .../SERVER/client/action/#
- Server client count is now published to .../SERVER/client (with the maximum client count, if applicable)
- Server states are now just represented by name (e.g. initializing, ready, stopping, stopped) and not number
- BBS errors are logged to sbbs/BBS/action/error/LEVEL (where LEVEL is the log level name, e.g. "critical" or "error')
- All server hack-attempts, SPAM attempts, logins, logouts, uploads, downloads, are published to sbbs/BBS/action/ACTION/*
- Chat pages are published to sbbs/BBS/action/page/node/#
- New users (on the terminal server) are published to sbbs/BBS/action/newuser
- Posted messages and executed external programs (on the terminal server) are published to sbbs/BBS/action/ACTION/CODE topic
- The event thread started/stopped status is published to .../SERVER/event

Yeah, the wiki will get updated soon to reflect/document all these changes

This function has been causing somewhat rare crashes (e.g. one per week on a very active system with MQTT enabled) on both Windows and Linux (see issue #495). The root-cause is still unclear (possible heap corruption?). This function needed more robustification anyway (see previous TODO comment), so I'm refactoring here to no longer use strListAppendFormat, which uses vasprintf (the function where the rare crash was occurring) and instead just use snprintf and strListPush. The total client_list that's published (as a single string/message) is now dynamically allocated (this was the point of the previous TODO comment). This may not actually fix the issue if there's a heap corruption occurring somewhere else in this function or the call-chain that's reaching here (usually from the web server).

Another change to mqtt_client_on(): don't incremented the server's 'served' counter until client disconnects (this is a past-tense statistic).

Also:
- mqtt_pub_strval() can now accept a null 'str' argument which is effectively the same as mqtt_pub_noval(), so the pub_noval() function is now just a thin wrapper around pub_strval().

- mqtt_startup() now clears the client_list topic and sets the client_count topic to 0.

When MQTT protocol version is >= 5, just log to the */log topic
(the level is a property).

This means you can't subscribe only to specific log levels when using MQTT v5
so I may re-think this, but I don't want to keep publishing every single log
message twice.

This can be useful when debugging issues (e.g. crashes) and an MQTT consuming/logging client wasn't connected at the time.

Yes, someone could have a local hostname of "node". <sigh>

This solves the problem of subscribing to "sbbs/+/+/+" and getting node status in addition to server status (very different formatted messages).

Also include a change for the singular-again "node" topic

Should fix the segfault reported by NeoArata_ in #synchronet

Also include singular/plural error/s in server status messages.

<server>/state is now published directly to the <sever> topic level (the "state" topic goes away).

Moved <bbs-id>/node# topics to <bbs-id>/nodes/# (allows better wild-card subscription per MQTT standards).
Publishing total node count to <bbs-id>/nodes rather than <bbs-id>/node_count

Introduced "publish verbosity" control (defaults to "High") - if you want nice human readable (and sometimes redundant) topic messages, leave this set to "High". To reduce traffic, set to "Low". It's expected that purpose-built Synchronet/MQTT clients should work equally-well with either setting, but when using generic MQTT clients/browser, "high" verbosity is nice. The human-readable node status is only published when "high" verbosity is enabled.

Publishing the BBS name to the <bbs-id> topic, the instance hostname (as configured in sbbs.ini) to the <host> topic. Makes the hierarchy much more clear when using an MQTT browser like MQTT Explorer.

This is more consistent with how these events are logged in a BBS-common log file in data/*.log.

This change also restores the server abbreviation to the error log entries that used to be there until recently.

Each Synchronet server is now its own MQTT client. This means there's no
longer any MQTT logic in the Synchronet "hosts" (e.g. sbbscon.c, ctrl/*.cpp)
and none needed for SBBS NT services (they'll "just work" with MQTT).

This also means that just about everything (except for nodes, spam and hack)
is now published per-server (in the sbbs/BBS-ID/hostname/server/ topic branch)
and if you want aggregated totals or client lists, you'll have to do that in
your own MQTT client or dashboard.

I also removed the publishing of thread_count and socket_count topics as
they weren't universally supported across all servers and are of questionable
value. They can be added back later if determined to be useful.

Should fix the new crash issues reported by Retros via #synchronet on irc.synchro.net

warning: passing argument 2 of ‘mosquitto_message_callback_set’ from incompatible pointer type

Now support subscriptions (e.g. recycle topics, node input topics) in SBBSCTRL.
This required a lot of search/replace and fun with circular struct pointers.

The only difference in the data/scheme is that the "error" topic (error log) is now under each server rather than each host. I don't *think* there are any other changes from the MQTT consumer side.

Still not done: subscribing (e.g. support for recycle or node-spy-input via MQTT) and NT services support.

This change also includes a cool feature that will prompt the sysop if there's a timeout (30 seconds) while waiting for servers to shutdown gracefully and giving the sysop the option to abort (Cancel) the wait (and shutdown ungracefully) or continue the wait (OK).

By moving mosquitto_will_set() call into mqtt_connect().

I'm still having the issue that the LWT is being published after mosquitto_disconnect(), even though I set the "force" parameter to false in the call to mqtt_loop_stop()