aboutsummaryrefslogtreecommitdiffstats
path: root/util
Commit message (Collapse)AuthorAgeFilesLines
* Merge 0.11->trunkKim Alvefur2021-07-031-1/+1
|\
| * util.ip: Fix netmask for link-local address rangeKim Alvefur2021-07-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | This may have mistakenly caused link-local addresses to be considered global. May have caused mod_s2s and prosodyctl check dns to behave incorrectly on networks using link-local IPv4 addresses. By my guesstimate, these are extremely rare. Probably minimal impact beyond a bit longer to establish s2s and some possible confusion from prosodyctl check dns results. Ref RFC 3927
* | util.format: Escape ASCII control characters also in extra argumentsKim Alvefur2021-07-031-1/+1
| |
* | util.format: Allow newlines but ensure following lines are indentedKim Alvefur2021-07-031-1/+1
| | | | | | | | | | This should a) prevent injection of text that looks like legitimate log lines and b) not mangle tracebacks.
* | util.format: Fix missing backslash in patternKim Alvefur2021-07-031-1/+1
| | | | | | | | | | Made the pattern match a longer range than intended, but with no effect since those characters are not present in the 'control_symbols' table.
* | util.prosodyctl.check: Reload unbound to ensure hosts.txt is ignoredKim Alvefur2021-07-031-1/+3
| | | | | | | | | | | | If unbound was initialized prior to this then the config change here wouldn't apply, and it will again think that 127.0.1.1 has been found in DNS.
* | util.format: Change formatting of nil values to avoid looking like XMLKim Alvefur2021-06-291-2/+2
| |
* | util.stanza: Export pretty printing functionKim Alvefur2021-06-291-1/+3
| |
* | util.stanza: Simplify and make pretty-printing look nicerKim Alvefur2020-11-071-29/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | I've had this color theme in a local debug module for some time and I quite like it. The colors are from the XMPP logo. Removes extra XML serialization implementation in favor of the standard one. Also removes recursive str=str..more string building. The new two-level gsub has the accumulator in C space so shouldn't be too bad. The inner gsub calls use no callback, so should be fast and not create all that much garbage. No serious benchmarking has been done, but who cares if it looks nice?
* | util.stanza: Remove Windows "support" (disabling ANSI color pretty printing)Kim Alvefur2020-11-071-11/+2
| | | | | | | | | | | | Always enable pretty printing if util.termcolours is available util.termcolours can be nooped out to disable pretty printing.
* | util.format: Escape ASCII control characters in outputKim Alvefur2021-06-151-1/+16
| | | | | | | | | | | | | | This should offer some protection against doing evil things to terminals. Doesn't protect against pure broken UTF-8 garbage however. See #734
* | util.dbuffer: Fix bugs, remove multi-char support (more complex than first ↵Matthew Wild2021-06-291-3/+3
| | | | | | | | | | | | | | | | | | thought) Character sequences could be split across chunk boundaries. Would require a bunch of code to make that work reliably. Only apply front_consumed on first chunk, and adjust buffer_pos accordingly.
* | util.dbuffer: Add read_until() methodMatthew Wild2021-06-291-0/+14
| |
* | util.prosodyctl.check: Collect options from all global pluginsKim Alvefur2021-05-271-0/+45
| |
* | util.prosodyctl.check: Format, sort option listings into canonical formKim Alvefur2021-05-271-8/+34
| | | | | | | | | | Makes merges and diffs easier to read and deal with, especially when using tools to gather options from the codebase.
* | util.dataforms: Define a integer + "max" datatypeKim Alvefur2020-09-181-0/+9
| |
* | util.prosodyctl.check: Ensure that libunbound does not check hosts fileKim Alvefur2021-06-201-0/+3
| | | | | | | | | | | | This fixes wrongly reported DNS problems on some distros where the hosts file contains an entry for the local machine, pointing at a loopback address such as 127.0.1.1 or similar.
* | util.prosodyctl.check: Deprecate legacy_ssl related optionsKim Alvefur2021-06-201-0/+1
| |
* | util.prosodyctl.check: Add support for checking Direct TLS SRV recordsKim Alvefur2021-06-201-1/+23
| |
* | util.prosodyctl.check: Fix for net.dns vs unbound API differenceKim Alvefur2021-06-201-2/+2
| | | | | | | | | | net.dns returns nil for NXDOMAIN, while net.unbound returns a table with zero items and various status fields.
* | util.prosodyctl.check: Add knowledge of the global-only 'use_ipv4' settingKim Alvefur2021-06-201-1/+1
| |
* | util.openmetrics: Use pack from util.table, detect appropriate unpack for ↵Matthew Wild2021-06-141-4/+5
| | | | | | | | Lua 5.1 (thanks sups)
* | util.openmetrics: Prettify format of histogram bucketsKim Alvefur2021-06-081-1/+1
| | | | | | | | | | | | | | | | | | "%g" turns 1GB into 1.07374e+09, which is a bit awkward for the bytes measurements IMO. Turning up the precision, at "%.17g" turns 0.1 into 0.10000000000000001 while "%0.16" gives 0.1, hiding most of those pesky floating point artefacts. Lua version 5.2 uses "%.14g" ( see LUA_NUMBER_FMT in luaconf.h.html ) so it seems like a sensible choice here.
* | Merge 0.11->trunkKim Alvefur2021-05-271-1/+1
|\|
* | util.stanza: add at_top helperJonas Schäfer2021-05-081-0/+4
| | | | | | | | | | | | This allows the user to detect whether the cursor is currently at the top of the stanza. This will be used by util.xmppstream to be able to port it over.
* | Merge 0.11->trunkKim Alvefur2021-05-161-0/+4
|\|
| * util.pubsub: Fix traceback if node data not initializedKim Alvefur2021-05-161-0/+4
| | | | | | | | | | This should not be possible, but a traceback indicating node_data being nil here was reported by Ron in the chat.
* | util.jwt: Use constant-time comparison with expected signatureMatthew Wild2021-05-131-1/+2
| |
* | Merge 0.11->trunkMatthew Wild2021-05-134-3/+21
|\|
| * util.xmppstream: Allow dynamically configuring the stanza size limit for a ↵Matthew Wild2021-05-101-0/+3
| | | | | | | | | | | | | | stream This may be useful for any plugins that want to experiment with different policies for stanza size limits (e.g. unauthenticated vs authenticated streams).
| * util.xmppstream: Mark bytes for stream closure as handledMatthew Wild2021-05-101-0/+3
| |
| * util.xmppstream: Reduce default xmppstream limit to 1MBMatthew Wild2021-05-101-1/+1
| |
| * util.set: Add is_set() to test if an object is a setMatthew Wild2021-05-101-2/+8
| |
| * util.startup: Set more aggressive defaults for GCMatthew Wild2021-05-071-1/+7
| | | | | | | | | | | | | | | | | | Testing has demonstrated that the default GC parameters are not sufficient to prevent runaway memory growth when running under Lua 5.2 and Lua 5.3. Setting the GC speed to 500 was tested on Lua versions 5.1->5.4 and did not display unbounded memory growth.
* | Statistics: Rewrite statistics backends to use OpenMetricsJonas Schäfer2021-04-183-209/+722
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The metric subsystem of Prosody has had some shortcomings from the perspective of the current state-of-the-art in metric observability. The OpenMetrics standard [0] is a formalization of the data model (and serialization format) of the well-known and widely-used Prometheus [1] software stack. The previous stats subsystem of Prosody did not map well to that format (see e.g. [2] and [3]); the key reason is that it was trying to do too much math on its own ([2]) while lacking first-class support for "families" of metrics ([3]) and structured metric metadata (despite the `extra` argument to metrics, there was no standard way of representing common things like "tags" or "labels"). Even though OpenMetrics has grown from the Prometheus world of monitoring, it maps well to other popular monitoring stacks such as: - InfluxDB (labels can be mapped to tags and fields as necessary) - Carbon/Graphite (labels can be attached to the metric name with dot-separation) - StatsD (see graphite when assuming that graphite is used as backend, which is the default) The util.statsd module has been ported to use the OpenMetrics model as a proof of concept. An implementation which exposes the util.statistics backend data as Prometheus metrics is ready for publishing in prosody-modules (most likely as mod_openmetrics_prometheus to avoid breaking existing 0.11 deployments). At the same time, the previous measure()-based API had one major advantage: It is really simple and easy to use without requiring lots of knowledge about OpenMetrics or similar concepts. For that reason as well as compatibility with existing code, it is preserved and may even be extended in the future. However, code relying on the `stats-updated` event as well as `get_stats` from `statsmanager` will break because the data model has changed completely; in case of `stats-updated`, the code will simply not run (as the event was renamed in order to avoid conflicts); the `get_stats` function has been removed completely (so it will cause a traceback when it is attempted to be used). Note that the measure_*_event methods have been removed from the module API. I was unable to find any uses or documentation and thus deemed they should not be ported. Re-implementation is possible when necessary. [0]: https://openmetrics.io/ [1]: https://prometheus.io/ [2]: #959 [3]: #960
* | util.prosodyctl.shell: Fix for different location of unpack in Lua 5.1Kim Alvefur2021-04-251-1/+2
| |
* | util.prosodyctl.shell: Fix for missing 'sep' arg to string.rep in Lua 5.1Kim Alvefur2021-04-251-1/+2
| |
* | util.prosodyctl.shell: Allow calling console commands with fewer shell quotesKim Alvefur2021-04-251-3/+2
| | | | | | | | | | | | | | | | E.g. `prosodyctl shell module reload disco example.com` becomes equivalent to `prosodyctl shell 'module:reload("disco", "example.com")`. Won't work for every possible command, but reduces the amount of shell quoting problems for most common commands.
* | util.xmppstream: Send <?xml?> declaration in same call as stream openKim Alvefur2021-04-141-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If network_settings.opportunistic_writes is enabled then this would previously have resulted in two socket writes, and possibly two packets being sent. This caused some issues in older versions of Gajim, which apparently expected the stream opening in the first packet, and thus it could not connect. With this change and opportunistic_writes enabled, the first packet should contain both the xml declaration and the stream open tag. Without opportunistic_writes, there should be no observable change. Tested with Gajim 1.1.2 (on same machine). Unsure if loopback behaves differently than the network here.
* | util.prosodyctl: Install plugins for current Lua versionKim Alvefur2021-03-161-2/+2
| | | | | | | | | | | | | | | | Fixes that luarocks defaults to installing the rock for its own runtime version of Lua. This only works with luarocks 3.x, it does nothing on 2.x as currently available from Debian.
* | util.promise: Switch order of parameters to join()Matthew Wild2021-03-251-4/+2
| | | | | | | | | | This saves awkward fiddlery with varargs and also echoes the signature of pcall/xpcall.
* | util.promise: all()/all_settled() pass through non-promise valuesMatthew Wild2021-03-251-22/+30
| |
* | util.promise: Add join() convenience methodMatthew Wild2021-03-251-0/+11
| | | | | | | | | | | | Usage: promise.join(p1, p2, function (result1, result2) [...] end)
* | util.promise: Add support for arbitrary keys in all()/all_settled()Matthew Wild2021-03-251-15/+27
| |
* | util.datamapper: Don't include empty unwrapped arraysKim Alvefur2021-03-241-1/+4
| | | | | | | | | | | | Since there is no way to distinguish an empty such array from a zero-length array. Dropping it seems like the least annoying thing to do.
* | util.datamapper: Fix error on attempt to coerce nil to somethingKim Alvefur2021-03-231-0/+3
| | | | | | | | | | | | Turns falsy values into nil instead of nothing, which ensures this function always has 1 return value, or table.insert({}) complains. Would still happen on some unexpected input, but that's actually a good thing.
* | util.datamapper: Deal with locally built stanzas missing xmlnsKim Alvefur2021-03-231-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So the problem is that xmlns is not inherited when building a stanza, and then :get_child(n, ns) with an explicit namespace does not find that such child tags. E.g. local t = st.stanza("foo", { xmlns = "urn:example:bar" }) :text_tag("hello", "world"); assert(t:get_child("hello", "urn:example:bar"), "This fails"); Meanwhile, during parsing (util.xmppstream or util.xml) child tags do get the parents xmlns when not overriding them. Thus, in the above example, if the stanza is passed trough `t = util.xml.parse(tostring(t))` then the assert succeeds. This change makes it so that it leaves out the namespace argument to :get_child when it is the same as the current/parent namespace, which behaves the same for both built and parsed stanzas.
* | util.datamapper: Factor out conversion from any value to XML stringKim Alvefur2021-03-221-29/+17
| | | | | | | | Since this was the last severely duplicated code left.
* | util.datamapper: Handle nested arrays or objects in arraysKim Alvefur2021-03-221-4/+22
| |
* | util.datamapper: Fix to skip parsing wrapped arrays that aren't thereKim Alvefur2021-03-221-2/+0
| | | | | | | | Turns out the unreachable error is reachable :D