Matrix.org hit some IO problems earlier this week, while largely a problem with our hosting provider, weāre spending a bit of time to make Synapse more resilient if the same thing were to happen again. This will mean the ability to shard the DB (by table) and spread the load so we are not so dependent on high performance from a given db box. Outside of that weāve been working on the final polishing of the room directory and getting the sqlite -> Postgres port script into better shape.
Not much for Dendrite this week as anoa is off at Ubucon 2019. But we had a few valuable bugs reported by the community, and a pressing reminder to get Dendriteās Monolith mode in as part of its CI.
Iāve not got much for this week, but the IRC bridge has been undergoing some serious refactors and changes for a larger release. Should be quite a big one when it lands :)
A PR for riot-web has emerged that adds support for search in E2E encrypted rooms. The PR is utilizing Seshat to perform event indexing and search on riot-desktop. While the PR is missing any sort of UI, it is in a usable state.
Take a trip down memory lane with the Riots of yesteryear at https://riots.im (note the āsā). Hosting every version of Riot Web released on GitHub š
RiotX: We have fixed quite a lot of issues during the stabilization sprint. We are now working on Sprint 4: read marker, report content, mark all room read, etc. FranƧois is changing the media/file picker and we will also be able to share elements from other apps to RiotX. We will schedule a release soon (tm) (should have happen this week, but has been delayed due to stabilization)
Polyjuice Client, a Matrix client library for Elixir, has a new release. There is now a short tutorial that will teach you how to make a simple echo bot with it.
Then:
Igor, a bot framework for Elixir, has had its first release.
Dept of Ping š
Rank
Hostname
Median MS
1
matrix.tetraodon.nl
323
2
aime.lesmatric.es
361
3
nerdsin.space
370
4
ru-matrix.org
371.5
5
c-base.org
377
6
fachschaften.org
430
7
linuxgl.ch
435
8
secureim.de
479
9
kif.rocks
484
10
aragon.sh
511.5
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Last week the spec core team said theyād start focusing on 3 MSCs per week. Those were MSC2290, MSC2176 and MSC1219. The first two have entered FCP, and MSC1219 will roll over into this week.
Thus, the 3 MSCs the spec core team will be focusing on this week will be MSC2199, MSC1219 and an up and coming security-related MSC. Join us in #matrix-spec:matrix.org for related spec discussions :)
There is also a new MSC, MSC2312 describing the proposed Matrix URI scheme. This is a remake of a much older proposal. The general idea is to make a standard for Matrix URIs: matrix:.
Coming up, improved room directory perf, ironing out wrinkles in the room upgrade UX as well as a major reliability boost in the sqlite -> Postgres db porting script.
Dendriteās latest hiatus has come to a close after the privacy work had taken so much of my time. Thankfully although PR review is blocked on the dendrite team, the community have continued to submit PRs and even review each otherās PRs (thanks cnly !).
Fixes this week have mostly focused around the CI. We finally got Dendriteās CI unborked (tāwas borked in a half-way transition between CircleCI and Buildkite), but itās now working and faster than ever. Weāre also working to add the Sytest test failure results to the top of the CI window such that people can see which tests failed and the associated Dendrite logs.
Progress of dendrite is tracked in Dendriteās milestones. Weāre currently aiming for #1 (Client-Side) Bot Hosting.
CI not running for community PRs - Buildkite hosts currently run multiple PRs from many different projects, so we canāt trust arbitrary code to run on them quite yet. Thereās a project in progress to run the code in a sandboxed environment (think VM or container) to lift this restriction :)
As a proof of concept, I have created a little Matrix client with Flutter for Web, named FluffyWeb. This client has only basic features but it shows the possibilities of Flutter for Web and it seems to work fine so far. The client has a responsive design and should work on mobile fine too. Check it out at: https://christianpauly.gitlab.io/fluffyweb/
(Not working in Internet Explorer - I recommend the AOL Messenger in this case)
Just released version 1.4.0 of the Ruby SDK, the main change in this release is the addition of a naĆÆve set of methods to replace the logger implementation. This should allow the gem to slot more easily into projects where existing logging configuration is already in place.
Dept of Ops š
1.4.0.Mania
With the release of Synapse 1.4.0, there was a rush to get packages and containers updated, the community are always fast, but we should acknowledge Anance for having his k8s images updated within a few minutes!
As always, referring to the projectās changelog before upgrading is a good idea.
KUDOS to this this project! I love that I can git pull, ansible-playbook -i inventory/hosts setup.yml --tags=setup-all without really checking and my toy homeserver is automagically updated.
Letās reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Following on from last week, the 3 MSCs the spec core team have chosen to focus on this week are: 2290 (3pid binding endpoints), 2176 (redaction rules), and 1219 (key backups), spurred on by finishing off the spec work for the privacy sprint, and cross signing.
MSC Updates:
News from 2019-09-20 09:00:00 until 2019-09-27 20:07:55.
This week we put out a release candidate for 1.4.0 which support a whole host of privacy features including greater control over interactions with identity servers, cleaning up redacted events and user meta data (like IPs and user agents) and warning users when they are using the default trusted key server (matrix.org).
Aside from privacy, the thing that is most exciting is switching on our solution for mitigating forward extremities build up by default. It should make a big difference for the CPU of servers in fragmented rooms.
matrix-corporal v1.6.0 was recently released to address an issue when used in conjunction with Synapse Single-Sign-On login flows (CAS or SAML).
Until now, matrix-corporal used to interfere with /login requests and demand that users always authenticate with a username/password.
Since v1.6.0, you can use matrix-corporal for automatic management of users/rooms/communities, while letting authentication happen through SSO (as provided by Synapse).
Iām also working on a maubot that accepts invites, announces them in a control room and then invites people from that control room when requested. That bot should be ready NWIM.
The slack bridge got an RC3 today and it is now LIVE on matrix.org!! This brings in a whole host of new changes like speedups for message processing, typing notifications and more reliable edits/reactions/replies. Since the 1.0 release required a migration of data files, we have made every effort to migrate integrations across. If you find that your slack bridge (hosted on matrix.org) is no longer working, please contact me
Work on Bifrost has resumed! Weāre doing our best to refactor and replace bits that were hacked together at the start of the year and really improve on reliability and documentation. The official matrix.org bridge awaiting our work on PostgreSQL before we can move it further, but the project is accelerating :) https://github.com/matrix-org/matrix-bifrost/tree/develop
Hello. I made a thing for using the Vim text editor for sending messages in Riot, and a friend suggested I share it here. Here it is: https://gitlab.com/MRAAGH/riot-vim#riot-vim
I have extracted Igorās guts, and have distilled them into a new client library for Elixir: polyjuice_client. Iāve also started working on a library of Matrix utility functions for Elixir that would be relevant to multiple components (clients, application services, homeservers, identity servers): polyjuice_util.
RiotX: A big work to stabilize the application and to implement little missing feature has been done. Also Two releases has been done this week. Please refer the the changelog for a (rather) complete list of what has been done (https://github.com/vector-im/riotX-android/blob/develop/CHANGES.md)
Also the read marker feature is in review. There are still remaining bugs which will be fixed before the merge.
I made a docker image for Synapse (again). However, this time it uses PyPy3. It is a drop-in replacement for matrixdotorg/synapse. Anyone is welcomed to test the impact of it on CPU utilization.
The matrix-fly-paper bot has been renamed to matrix-fly-swatter. On reflection, I realized the original goals for the bot were too ambitious and the project has become an exercise in becoming familiar with the client<->server api. The planned features have been cut back to simply automating sending m.room.server_acl events to several rooms at once. At a later date, planned features may be expanded, but there are other projects Iād have more fun working on in the meantime.
Dept of Ping š
Itās the section where we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median ms
1
matrix.tetraodon.nl
294
2
ufc.tu-dortmund.de
331
3
fachschaften.org
353
4
matrix.okeso.net
377
5
poddery.com
392.5
6
privacytools.io
422.5
7
dodsorf.as
433.5
8
linuxgl.ch
459.5
9
matrix.vgorcum.com
471
10
wolkenplanet.de
492
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
The Spec Core Team is going to try and select 3 MSCs to prioritise per week to work on, in order to not stretch resources too thin and to communicate clearly what it is weāre actually up to. More details to follow.
Otherwise this week has been continuing to iterate on existing MSCs, as well as updating the MSC template to rename the āTradeoffsā section to āAlternativesā, and to get rid of āConclusionā which was always just an unnecessary repeat of the introduction.
Dept of Servers š¢
Construct
Notes from Jason:
Construct did a round of performance work and pursued some of the easier opportunities with the most effective returns. At present, the biggest performance bottleneck Construct experiences is with I/O, specifically latency from accessing disks in a random pattern. This week a few open opportunities for prefetch pipelines were taken, allowing the server to stream data directly from the disk to the client with minimal buffering. This was implemented in several places, mostly for the /media/ system, and also for the /members lists, improving the performance of requesting the members of a room like #whalepool:ericmartindale.com .
There is one area though that requires a special mention for being an order of magnitude more impactful than any other area where a pipelined stream was placed. We have analyzed the way Riot conducts a room change, and by the simple placement of a prefetch operation when the server receives an im.vector.breadcrumb_rooms, which comes early during the room change, Construct is able to populate the roomās timeline significantly faster. That has a compounding effect, as the browser initiates several requests based on (and after) the received timeline data- meaning those responses come significantly faster.
Synapse
This week, lots of hacking but not much in the way of news. Weāve been focusing on more privacy improvements as well as polishing up the SAML integration.
Next week, expect a wodge of privacy related features to land.
The Matrix Slack bridge has had another release candidate this week. 1.0 RC2 is out now. Weāve rewritten the codebase in Typescript and made many changes, and need your feedback before release so please do not hold back on testing it! The 1.0 release is slated to arrive sometime next week :)
Presenting famedly-email-bridge, a new email bridge! It is still early in development but can already send and receive emails.
Features so far:
Receive emails by a prefix (e.g. matrix_sorunome@myserver.com, matrix_otheruser@myserver.com)
Send emails with said prefix
Receiving emails parses plain body, html body and attachments
Sending emails collects all messages for a given amount of time. After that it creates an email based on that! Supports edits, redactions and files (image, video, audio, file)!
Things planned for the future:
Route emails to channels based on threading (reply-chain)
Send emails by making a new channel and inviting an email ghost and just sending messages!
My other news that I forgot to mention is that I have a branch of the IRC bridge that runs off postgres, rather than flat file storage. Itās much less memory/cpu intensive and might even offer a bit of a speedup :)
For ref: The current room/user store files for the freenode bridge take up 250MB when stored to disk. The existing storage system works on the principle of loading the whole datastore into memory and periodically saving to disk. Hopefully by not requiring the bridge to load all the state into memory, there are savings to be made.
mautrix-telegram v0.6.1 has been released, no changes since the release candidates two weeks ago. mautrix-telegram v0.7.0 is scheduled to be released within 6 weeks.
Also, thereās a WhatsApp business API bridge using Twilio in development, though Iām guessing most people wonāt have any use for that.
Riot Web worked on the last remaining bits of the current privacy sprint. We are planning to do a release soon that includes this privacy work from the last few months. We are also working on a few improvements for first time users around the create room dialog. We also shipped several patch releases (1.3.5 and 1.3.6) to fix high priority bugs.
Riot-iOS has made some progress on the privacy sprint. We fixed some major issues with VoiceOver. Those fixes will be released soon in a hot fix release: 0.9.5.
A release has been done (0.5.0) with login with SSO support. There is also a new āno networkā indicator, which is a bit bugged, we will fix that.
Stabilization sprint is still ongoing. We already have fixed quite a lot of issues.
FranƧois is still working on the read marker and is also fixing issues related to the permalinks.
It might be interesting to some of you that the F-Droid repo for RiotX is finally working again.
There was an issue with buildkite which in the end led to the situation that the built apk had a lower versionCode as the apps already in the repo. The repo was then āintelligent enoughā to no offer it as update.
benoit was able to resolve that so the repos are updating again.
Dept of Ping š
New!! A new section where we will reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
In fact theyāre having a standoff between Matrix, Mattermost, Rocket.chat and Slack.
Weāre eager to see the results of this trial, the feedback will be very valuable for everyone regardless of the final outcome. (Though of course, we are optimistic..!)
This week sees us complete the last items on the privacy sprint and we expect a release to land early next week. Aside from that OpenTracing is now live, the configuration tool is ready to go (modulo the new Synapse release) and weāve been trialling out room directory improvements.
Next week expect Synapse 1.4 to land which will contain privacy sprint support as well as a host of bug fixes and perf improvements.
Hello Twimians! After many weeks of creativity, pain, and slack, the bridge team are proud to present our first release candidate for the slack bridge. This release brings in a ton of new changes that should massively improve the slack<->matrix experience. The headline features are Postgres support, Puppeting, RTM (websocket) support as well as an entirely refactored codebase written in Typescript.
This release has made heavy changes to the schema of the slack bridge and as such we are not recommending that anyone upgrade their existing slack bridge instances to 1.0.0-rc1. A migration script is currently being worked on, but for the time being existing installs should not be upgraded to 1.0.0 release candidates.
Hi all! I havenāt been around much because our homeserver chugs a little when federation is enabled, but Iāve got a few updates to share about Scylla, the Elm-based web client for Matrix:
There have been a few optimization changes. The naive Elm approach leads to recomputing room names and usernames on every keystroke, which causes huge performance issues in rooms with many users (like #freenode_#haskell!). The new versions of Scylla avoid recomputing room names and message HTML, leading to much better performance in large rooms and rooms where a lot of history is loaded.
The recommended fixes in the spec regarding flickering have been implemented, and messages should no longer flicker.
ām.noticeā and ām.emoteā events are now supported
There has been a minor visual update to make the interface look a little more consistent and professional.
We have released FluffyChat 11.16 with minor bug fixes and updated translations. A nice new feature: Messages which contains emojis only are now displayed with a much bigger font. š
Continuum desktop client written in pure Kotlin, version 0.9.23:
Include Elliptic Curve module to be able to connect to HTTPS servers using ECDSA
Removed dependency on kotlin-reflect and tornadofx completely and make packaged size about 5 MB smaller, which is a 10% to 20% reduction, depending on the target platform and package format
In addition to automatic scaling when HiDPI is detected, font size scaling is added. Press Ctrl and + or - to increase or decrease the font size. Font size scaling is not limited to texts, components with their size defined relative to the font size also gets scaled.
When a image message is clicked, a viewer is opened to provide a closer look. Now the updated viewer is opened in new windows, multiple images can be viewed at once, and interaction with the main window is not blocked. Images also gets scaled when the viewer window is resized.
Removed borders and backgrounds from 3 buttons and give them a flat appearance.
Iāve been working occasionally for the past months on a basic matrix web client called Brawl, focusing on performance, offline usage and working on my old phone. I recently got it in still limited, but somewhat useful state. Unsure how much time Iāll have to keep working on this regularly, but wanted to share it here anyways. Check it out on https://github.com/bwindels/brawl-chat, thereās a GIF of it in action š
Riot Web
From the team:
Weāve made good progress on our first FTUE (First Time User Experience) project: āImprove add & create roomā, with only the makeover of the create room dialog remaining. Next week more FTUE projects, and we hope to get some progress on cross-signing, barring any other distractions.
Riot iOS
From the team:
Finalising privacy
Release 0.9.4 with support of the new Riot configuration link to quickly customise servers on the authentication screen.
Riot Android
From the team:
Two minor releases have been made (0.9.5 and 0.9.6), to allow auto configuration of the homeserver in the login screen with a configuration link, and to fix an issue with SSO using Google account.
Still working on privacy
RiotX (Android)
From the team:
Working on stabilization. We already have fixed some issues.
FranƧois is still working on ReadMarker
Benoit is working on saving draft (storing unsent messages)
Also implementing SSO login and M_CONSENT_NOT_GIVEN error
Had some free time and ideas this week, so Iāve improved my release tracking bot to now correctly handle repos mixing full releases as well as tags (lightweight and/or annotated). GitHubās API really sucks in the tags department, so this required a rework to GraphQL queries instead, and itās looking good on that so far.
The bot currently still only handles tracking all starred repos of a given account on GitHub, but Iām working on GitLab support, as well as support for tracking arbitrary repos. And a provisioning API is also somewhere on the roadmap,
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
This week privacy items have dominated, but weāve also been edging towards getting our room directory improvements out, shipping a new configuration creator and making further progress on the federation side bus project to improve performance of smaller instances.
We also deployed open tracing on matrix.org, which will help a lot in tracking down all sorts of strange behaviour in the wild and fixing some bugs in the sqlite -> postgres migrator.
Next week weāll land the room directory improvements, much of the privacy work, and have the configuration creator ready for testing.
Construct
Jason:
Construct added user room listing support so that you can find out what public rooms a user is a member of should they choose to not be invisible. To find out the list of rooms for a user, simply open the room directory in Riot and type a user mxid starting with @ā¦
Construct is implementing the concept of invisibility for users:
The exact mechanism for invisibility is not completely finished but there are obviously some limitations for this feature:
Only users who are local to the server can set any kind of invisibility. Remote users are always fully listed. One day when the spec has āuserās roomsā we would be glad to adhere to an m.user.invisible-etc configuration state event in such a room
Since there is no federation support for userās rooms list publication, the rooms list for a user is limited to what the server knows (rooms it has seen the user in).
Construct is eager to implement any MSCās regarding user room and group membership listing over the federation.
Construct is written for maximum performance in C++, and is developing rapidly - join #construct:maunium.net for more info.
matrix-media-repo updates for quarantining and purging media
matrix-media-repo has received a bunch of updates regarding quarantining and purging media - see the admin docs for more information. Thereās also an s3 upload bug fix if youāre using that.
lorico has been rewriting my old GitLab bot to python/maubot: https://github.com/maubot/gitlab/pull/23. Progress on that is looking very good and itāll probably be ready within a week or two.
I decided to make a minor mautrix-telegram release with some small fixes, since v0.7.0 will take a while still. v0.6.1-rc2 is out now, the full release will probably happen next week if nobody finds any problems. Main changes:
Invalid reply fallbacks sent by Riot web in edits are now ignored
User and portal sync errors are now caught so that the sync command wouldnāt fail if one telegram chat disappears.
Dept of Clients š±
Riot Web
Tidy update from Bruno this week:
This week we were doing the last bits of the privacy work, and fixing bugs and adding a format bar in the new composer (still behind labs flag).
Weāve also been planning the First Time User Experience work, which will start at full speed next week. This is an effort to address some issues that new users hit in the first days of using Riot. See https://github.com/vector-im/riot-web/projects/16#card-25137120 and Matthew talking over the details at https://youtu.be/JYsiHSL0lEA?t=710. Letterboxes in the left panel will likely only be removed as part of the new community project though, as we need parts of the new community UX to keep things usable for power users.
Also, if youāre using https://riot.im/develop please enable the new composer in Labs, it works well and deserves testing.
Continuum desktop client written in pure Kotlin, version 0.9.20:
Updated login page appearance
Added validation to user ID and server address fields
The server address gets automatically filled with https:// followed by server name when user ID is being typed
After signing in, the token and server address are automatically saved
Added progress indicator when being signed in. It doesnāt block user from controlling Continuum, pressing Esc will cancel the signing-in process.
Speed up application start-up by drawing the window and opening the database in parallel. Now the window typically appears within half a second with its size and position on screen restored to be the same as the last time it was opened. The most recently used accounts get loaded and automatically filled within a second.
Added drop shadow effect to scroll-to-latest-messages button to increase contrast
Fix: Always retry synchronization after getting a network error. Previous versions didnāt retrieve new messages in a timely manner when the network is unreliable. A delay is added if the error is not a time-out to avoid stressing the homeserver.
Fix suspend/hibernate detection. Synchronization is restarted automatically after the computer resumes from sleeping. The reason is that any previously opened connection is unlikely to be still alive, opening new connections reduces delay.
Fix incorrect interface declaration that caused login to fail
Please let me know what you think of the separation of personal and public chats! Currently itās implemented using a bottom navigation bar, but tabs could also be an option (which is closer to what WhatsApp does)
t2bot.io has surpassed 300k monthly active users (about 35k daily) š
roughly 250k are (bridged) from Telegram, the rest is from Discord. (thereās some very small numbers which include the bots, webhooks, and other tiny bridges, but together they form less than 100 users)
Inspired by a question in the Matrix HQ channel, Iāve built a (rudimentary) room list viewer at https://matrix-rooms.cryto.net/ - you can just enter a homeserver (hostname) and see all the public rooms on it, without needing to log into anything. This might be useful for showing other people what kinds of rooms exist on Matrix!
Dept of Blockchain š¤·ā
OI Chat, Matrix based chat uses blockchain for authentication
OI Chat has been updated and now connects blockchain users on Blockstack and EOS (and the rest of the matrix universe): https://www.producthunt.com/posts/oi-chat-2
The sever has no passwords or requires permissions by other social account providers, it verifies the authenticity of the user using properties of the blockchain IDs, it names registered on the blockchain.
Dept of Ops š
Matrixmon
Matrixmon new release 0.4.0, featuring Docker image hosted on Docker Hub and an option to specify custom config file location.
matrix-fly-paper is a new bot project intended to help automate server and room moderation. Itās currently not ready for any kind of use, as Iām trying out serious TDD and attempting a functional approach to break from my OO training. Feedback is appreciated, stop by #matrix-fly-paper:lost-angles.im to talk shop
About a month ago we started selling merch at shop.matrix.org, and WOWEE ZOMG we sold a lot of stuff! If you are happy with your purchase Iād love to hear from you - share photos of where you put the stickers or you happily wearing the t-shirt in #twim:matrix.org and Iāll make a little collage.
If you are not happy with your purchase then please message me at @benpa:matrix.org and I will try to make it right.
Final thoughts š
You should complete this EU-related FOSS survey, and if you wanted to say nice things about Matrix that would be fine by us.
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
I made a few spec proposals: MSC2244: Mass redactions and MSC2246: Asynchronous media uploads. Mass redactions are useful for fighting spam (and bridging telegram deletions) more efficiently. Asynchronous media uploads would make it possible for bridges (like telegram) to handle messages synchronously and thus keep message ordering without blocking all messages behind some media upload.
Dept of Servers š¢
Synapse
Erik reported:
Busy landing the remaining items for the privacy project.
Reviewing and landing the room stats work Oliver has been busy with before he finishes his internship this week (thanks Oliver!).
Jorik has been doing the finishing touches to the Synapse installer (which looks awesome fyi).
Weāve also been looking at a basic message retention implementation in Synapse.
Finally, Hawkowl had a lot of fun at Kiwi Pycon last week (woo keynote!), and is now back and looking at improving performance of outbound requests.
Construct
Jason from Construct:
Construct added support for server notice rooms. As the server administrator Iāve placed quick access to the server control room for myself in this section.
Construct has also added support for the server-side commands \\read <room-tag>, for example \\read m.low_priority will mark all low-priority tagged rooms as read. The command \\read * also will mark all rooms as read.
Also, we now allow users to configure whether they transmit typing events by setting state events in their home-room. If a user creates an event with the type of ircd.typing.disable.send and the state_key of the room_id, typing events will not be sent to that room for them. Further variations exist, such as ircd.typing.disable.sync to prevent the client from receiving typing events for a particular room or all rooms.
For all those who want a new emailbridge, I have great news for you! The new bridge supports sending and writing emails, using multiple rooms for multiple email-addresses. In addition you can use markup to improve the style of your emailbody, which gets translated to HTML. Here you can find the repository: Matrix-EmailBridge
The Slack bridge has had a whole week of care applied to it. Weāve switched the bridge entirely over to Typescript, and overhauled large parts of the codebase to make future work easier (for ex, using the Slack node library rather than request calls). If thatās not exciting enough, weāve also added support for the RTM API, which means the bridge can use websockets when talking to slack. This gives us nice things like typing updates, and the ability to deploy without needing a webserver to listen for events. Release 1.0 is imminent, but before then we are hoping to land Postgresql support so that the bridge can scale up to meet the high demands placed upon it.
A new version has been pushed to F-Droid, Google Play and TestFlight!
ā ļø This update requires a reinstall or clearing of data ā ļø: Because of some database
changes which were not that easy to migrate, youāll have to reinstall the app or clear the data.
Expect the new version to be on most platforms in 2 days, and on Google Play
in a few hours.
Changes
Add a dark theme!
Add basic settings page
Add ability to change your own display name
Allow multiple lines while typing a message
Render line breaks in messages
Fix issue where homeserver setting was overridden by the user domain
Mark new messages as read while chat is opened
Fix chats being sorted by your own display name change
Fix multiple issues regarding the member count of room being null
Use Flutter beta, which means that you can paste in textfields again!
Provide builds of self-contained executables that have no external dependency. The Linux version has been done. AppImage is used, producing a binary of about 50 MB.
Adjust appearance. Remove alternating row high-lighting in list views. Remove extra padding between messages.
Thanks to Roland Pallai, Quaternion master branch is now able to use customised fonts and sports a faster timeline, better copying and a few other improvements. Settings dialog window is coming.
Initial code dump of Seshat, a Matrix event indexer/database that should enable search for encrypted rooms in riot-desktop. The dump contains a rust library as well as node bindings. The repo can be found here.
Apparently there are no ARM docker images for synapse, so I built a multi-arch synapse docker image. Supported CPU architectures include armv7, aarch64 and amd64.
Thereās a useless new maubot plugin that lets you run python and soon shell scripts from Matrix. Not intended as a public bot as it doesnāt have any sandboxing, but it does have a āuserbotā mode to run it under your own account so that you can use it in any room.
Slightly mean to call your own bot useless :(
Final thoughts š
If youāve suffered from invite spam lately, you can solve the problem by following these instructions:
Itās CCCamp 2019 this weekend. Tonight, anoa will be leading an āInstall Synapseā workshop in the Librehosters Village- keep decentralisation growing!
Dept of Servers š¢
Synapse
Neil:
This week weāve been working on getting our brand new room directory implementation ready, creating a new Synapse installer, and creating a TURN server testing tool. Weāve also been pushing on with the privacy project and fixing up some bugs and perf issues.
Next week itās all about getting the room directory out of the door, shipping the installer and continuing on with the privacy project. We are in the early stages of a project designed to reduce the load caused by outbound SSL connections, it is looking very promising. Weāre also going to give the sqlite->postgres porting script some love as well as working on some message retention ideas, and also ways to reduce the meta data we store.
the adding portal rooms to user-specific communities feature that appeared in mautrix-whatsapp and mautrix-hangouts last week is now also available in mautrix-facebook
The gitter bridge is back to working order again today! Weāve made changes so that Gitter side admins can now issue a !matrix kick|ban name command to evict users who are causing issues, rather than relying on reports or matrix side admins.
The matrix-appservice-bridge library also saw its 1.10.0 release this week, which has experimental support for bridge errors and also has a bugfix related to room upgrades.
Finally, the discord bridge has very nearly switched to Travisās bot SDK stuff. We are very close to a release, so please stand by :)
Dept of Clients š±
Riot Web
From the team:
Work on privacy.
Weāve also been continuing the work on the new message composer implementation. Features are still missing, but this has now landed behind a labs flag on /develop. Weād be especially interested in feedback whether typing with an IME (Chinese, Japanese, ā¦) works better than before.
Riot iOS && Riot Android
From the team:
Working on privacy:
Working on identity server v2 API
RiotX (Android)
From the team:
Read receipt work has been merged to develop. We will release next week, after a small UI iteration
If you would like to support me, you can do so via Liberapay and Patreon!
āmatrixcliā, new client enables Matrix as a unix util
Very interesting project: matrixcli is a versatile command line client using the matrix-python-sdk. Itās not a conventional IM client, rather it exposes access to Matrix rooms so you can pipe to them as you would other Unix utils.
Neoās refactoring is almost done, with the latest changes being css-modulesifying the old stylesheet
Weāre also looking for nice domain name suggestions, to house the soon to be redesigned project page
The GSoC is soon coming to the end this year and Iām happy no announce that receiving of the encrypted messages PR is here! https://github.com/quotient-im/libQuotient/pull/346
A significant amount of work still needs to be done (session storage, sending messages, device handling, attachments). And Iām interested to implement it after the summer period since Iāve already dived into the subject :)
Hereās the small demo of the receiving messages process: https://youtu.be/gGykLh4mVDg
Dept of SDKs and Frameworks š§°
Matrix-ClientServer-API-java
Jojii created a new Java-based implementation of the Matrix CS API: Matrix-ClientServer-API-java, they call it āA small and simple java API for the Matrix ClientServer Protocolā.
For people looking for a rather flexible ansible role just for synapse, https://gitlab.com/famedly/ansible/synapse got your back. You can use it to deploy synapse with pip or docker, depending on what you prefer for your host. Itās an actively maintained fork of https://github.com/matrix-org/matrix-ansible-synapse, which has been archived recently and was unmaintained for a while before that. Itās just been updated to v1.3.1 too.
This increases the number of services/protocols that the playbook can setup bridging for to 8: Telegram, Whatsapp, Facebook, Google Hangouts, IRC, Discord, Slack and Email.
Learn more in Bridging other networks.
matrix-docker-ansible-deploy gets support for all Synapse config options
Just built a quick maubot plugin to add a /health endpoint to maubot, so that database failures and other brokenness can be tracked and dealt with appropriately.
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
privacy; IS and IM permissions and management. Hashed lookup API. Metadata cleanup (GCing redactions; deleting media; history retention); TURN management
aggregations paused
FTUE next up
Cross-signing
Immutable DMs
ā¦then communities
RiotX
0.3 came out last week - handles DMs; lots of polishing; reduces the APK down to ~10MB!
Bridges
moving stuff into proper managed infrastructure
bifrost should be back soon?
gitter kept breaking but weāre actually working with their team to fix that
working out how to automate deploying them on modular hosts
Progress for Dendrite this week has been a bit starved due to ongoing privacy work from the backend team. However it continues to move forwards with the excellent help of cnly reviewing and merging PRs.
We had 3 authors have pushed 5 commits to master and 6 commits to all branches. On master, 14 files have changed and there have been 298 additions and 121 deletions.
Also this week was some work from cnly on refactoring gomatrixserverlib and exposing some more of its internal constants to deduplicate code between itself and Dendrite.
Finally, we had some more work on an unmerged PR from cnly, which adds some internal query APIs for which servers are currently joined to a room, for use by other components later down the line.
No new Sytests are passing this week. Weāre still currently at 172 tests passing out of 761 tests in total.
Matrixmon updated
Matrixmon, the perl C2S testing utility has been updated to v0.2.0 with support for SSL connections.
Synapse
Synapse 1.3.0 was released - check out the blog post.
contains performance improvements to reduce disk I/O and reduce RAM usage. Weāve been running it on matrix.org for a week or so and are really pleased with the results.
mautrix-telegram got some fixes to bugs that appeared during the move to mautrix-python. It also got a manhole similar to the one in synapse, which gives admins access to an interactive Python shell inside the bridge while itās running.
The main part of the manhole (server and repl) is in mautrix-python, so you can expect the feature to appear in mautrix-facebook, mautrix-hangouts and possibly maubot in the future.
mautrix-whatsapp and mautrix-hangouts can now automatically add rooms to a user-specific community to help with filtering. mautrix-facebook is also getting that feature in the near future.
mautrix-facebook now has a search command, which means you can now actually initiate conversation with facebook users.
Dept of Clients š±
Riot Web
From the team:
Privacy work continues
STUN fallback server for assisting with voice & video call negotiation (only used when your homeserver is not configured with itās own TURN server) changed from Google to Matrix.org server and a prompt has been added to request permission before using the fallback server
Many small steps towards supporting user choice of identity servers and integration managers, including no identity server at all
Users can now download artifacts from my gitlab account since I got an SDK container from CoDerus running, cross compiling to Sailfish-RPMs in the Gitlab-Ci.
But my programming progress on master looks a bit silent because I swap to matrix-nio.
Rewrite the list view partially to handle lists with several types of items more efficiently. The message list has text messages, images, membership updates, etc. Each type needs to be rendered differently, but items of the same type could reuse GUI components when they scroll into and out of view.
Since the coming 1.3.0 release is bringing along a whole bunch of perf improvements, (and I want to test them out on my IO starved setup) Iām building K8s optimized images of the RCs. Only the debian version of the image though.
They expanded:
Itās basically the regular docker image, only with some changes to make it fit nicer in Kubernetes
And thereās no Helm chart at the moment, was poking at that for a bit but took a break from building one as I - back then - had trouble with the amount of data that had to be generated
You can only have one Synapse pod at any given time, as itās not scalable. Though you can scale it with workers (as I do myself)
Database-wise, Iām personally running three node HA clusters using Stolon
We run a public instance on Permaweb.io for everyone to join. Weāre a large community of developers focusing on the decentralized web, with lots of channels dedicated to IPFS. In fact, we even host Riot on IPFS! We make full use of Matrixās bridging capabilities, to Discord, IRC, Slack, Gitter and Telegram too! Come join us at #general:permaweb.io or register at https://riot.permaweb.io/ (or any other compatible client) and say hi!
Dept of Bots š¤
Notification/Remind-Me-Bot using matrix-bot-sdk
Anton (@antonivan:matrix.org) is working at Matrix Towers this summer for work experience. Heās been learning about Matrix and matrix-bot-sdk:
This week I made a Notification/Remind-Me-Bot using Travisās bot sdk, it took me 1 and half days, and is able to - set reminder, check reminder, cancel reminder, set interval reminder, cancel interval reminder. It is also able to store multiple reminders and cancel them using the numbers 1 - ā . Here is a link to my Github repo - https://github.com/joakimvonanka/Matrix-Remind-me-bot
Dept of Event Videos š¹
Matrix talk @ FrOSCon
Olegās talk at FrOSCon was apparently really well received:
Video recording from the Matrix talk @ FrOSCon.
Video is in German slides in English.
Heās also thinking of writing a guide to creating presentations:
If someone is interested on an efficient way of creating presentation using only open source tools Iām planing to write an article on that.
Small spoiler: matrix is one of them š
Final Thoughts š
This week, I have been mostly thinking aboutā¦ merch. We blew the cache on stickers, so if you are waiting for stickers specifically and have not heard from me, that is probably why. Please contact me with any questions!
Thatās all I know š
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Yes, itās true: Official Matrix Merch is available now. There are stickers, t-shirts and hoodies all available now, go and claim yours from https://shop.matrix.org.
I will be doing a talk on Matrix this weekend (tomorrow) at the annual Free Open Source Conference (FrOSCon) in Sankt Augustin (Germany).
If you are around - come by and get some awesome [matrix] stickers! =)
This weekās stats for Dendrite are in! 4 authors have pushed 11 commits to master and 11 commits to all branches. On master, 48 files have changed and there have been 676 additions and 146 deletions.
Very old PRs from CromFR have been updated and finally merged! These lay the groundwork for providing filters across various API endpoints (used in search, /sync, etc)
We also have a number of PRs that were moved forward but not merged. Add auth fallback endpoint (our oldest PR at the moment!) which we mentioned last week is so very close, but not quite there in time for this TWIM.
In terms of passing Sytests, Dendrite now passes 172 tests, up from 167 last week. This is out of 761 tests in total. So just a small bump (0.6%), but note that things that are crucial for Dendrite development at the moment, such as federating, only cover a small subset of tests, so each passing one can bring us much closer to getting Dendrite usable day-to-day.
Synapse
From Neil:
The big news this week weāve landed some massive DB improvements (https://github.com/matrix-org/synapse/pull/5706) which should make message sending on matrix.org (and any other server) feel noticeably snappier. :) Itāll be in the next release.
Work continues on our installer to make it easier to configure Synapse and weāll be looking for some community feedback rsn. The room directory revamp is now very close and we hope to have something live on matrix.org in about a week.
Finally weāre working on improving the efficiency of smaller instances sending messages into large rooms, weāre still at the design stage, but it will make a huge difference for anyone self hosting.
matrix-room-directory-server ( #matrix-room-directory:t2bot.io ) is less experimental than the key server but is still very early days. Currently it only offers the ability to manipulate the federated public room directory for your server, but in future it is planned to be its own standalone directory server (room aliases without having to run a whole homeserver). Check it out by searching the t2bot.io room directory from your client.
Dept of Clients š±
Continuum progressing!
I love getting updates from yuforia - theyāve been consistently working on Continuum for some time, and by increments are making a great client.
Reuse the ListView of messages across different rooms to reduce memory usage
(Experimental) Remember and refocus the last read message, making it easier to go through all unread messages while switching chat rooms freely.
Renaming of QMatrixClient to Quotient has been finally merged to the master branch; Quaternion master uses it from now, too. Packagers are welcome to make test builds and report bugs in #quotient:matrix.org. libQuotient 0.6 beta is coming close now!
Riot iOS
Test on iOS 13 Beta. Beta 6 fixed most bugs discovered on Beta 5
Released 0.9.2 (Waiting for Apple review at the time of writing)
mautrix-facebook, mautrix-telegram and the tulirverse
tulir has been making big updates to two of his bridges:
mautrix-facebook can now bridge formatting, mentions, replies and reactions in both directions.
There is also a logout command now.
mautrix-telegramās switch to mautrix-python is nearly finished (i.e. it didnāt cause any errors for the past few days when testing in production). The main reason for the switch is using one Matrix library for all my python bridges. It also means the bridging code like double puppeting and command handling I shared between mautrix-facebook and mautrix-hangouts is now also used in mautrix-telegram.
Visible changes directly caused by using mautrix-python:
Logs are now colorful.
Python 3.5 is no longer supported.
The bridge will refuse to start without access to the base config file.
Other changes that happened during the switch:
Telegram āSaved Messagesā can now be bridged even when using double puppeting.
Mentions on Telegram are marked as read when using double puppeting (messages were already being marked as read, but mentions werenāt).
Also, this actually happened last week and the week before that, but anyway: Iāve moved the CI and docker registry of my active projects to a self-hosted GitLab at mau.dev.
Specifically, the CI/docker registry for all four mautrix bridges and maubot and automatic builds for gomuks have been moved. My maubot plugins also have automatic .mbp builds in the CI. For the docker registry, prepending dock.mau.dev/ to the existing image names will work.
The old places (docker hub, dl.maunium.net) wonāt get new builds anymore. The repos on GitHub are still the ācanonicalā repos, but theyāre mirrored more or less instantly with maumirror.
He adds:
mautrix-telegram will probably get some sort of history filling in the near-ish future
Also,
Iām going to add some way to put bridged rooms into personal communities for filtering purposes. Not yet sure if itāll be fully built into the bridges or some kind of an external script
The configuration process (users, rooms, mappings) is quite manual, but itās simple and has worked well for me for the past year and a half.
Iāve only just polished it up a bit, released and integrated with the Ansible playbook, so others could benefit from it too.
I started a āmatrix-xmpp-filterā project. Itās like matrix-ircd, but with xmpp instead of irc. It can also filter messages by weekday, time and regular expressions. Target audience is Sailfish phone users, but it could be used with other xmpp clients too (no xmpp MUC support required). https://k2c42.dy.fi/matrix-xmpp-filter.git/#matrix-xmpp-filter:ellipsis.fi.
Currently working on a Puppet module to manage Matrix Synapse installs (both directly and through docker), not quite ready for prime-time yet but watch this spot.
I develop matrix Bot, which converting voice messages to text. It use Yandex Speech API and Yandex API cloud (some as aws) for temporary store voice-data before converting. https://github.com/progserega/voice2textMatrix
Bot have such logic:
user add bot to room (for example room with whatsapp bridge users - now bridge support adding bot to such rooms)
bot listen room, and when get voice-message - send it to Yandex-cloud for translate (now support only Russian language).
When translating is success - bot get result text and show it in room as notice, such: āUsername said: textā
Bot also receive some command, which allow disable it in this room, or disable translating for user, which send command..
Seeing this, I wondered, āy tho?ā, luckily progserega was able to explain with a graphic! They say a picture paints a thousand words, and itās clear from the conversation below why heād want to have speech-to-text capabilities.
Category Atom Feed
