Search

Lemmy Cafe will be having its database upgraded.

Reasons

• PostgreSQL 17 has been released and the changelog is promising a lot of IO improvements. Lemmy sure could use it given the constant stream of small events flowing in.

Plan

• Point nginx to the maintenance page
• Shut down PostgreSQL 16
• Run the upgrade tool
• Start up PostgreSQL 17
• Point nginx to lemmy

Expected downtime

About an hour, if things go well. More if not so.

Will try to keep the maintenance page updated.

Here's the timezone converter.

We’re aware of an issue affecting our Lemmy service (pawb.social) that seems to be causing users to be randomly logged out. We believe we’ve identified the issue thanks to a bug fix in a newer version of the UI code.

We’re needing to verify there are no other breaking changes for the upgrade process, and expect to perform maintenance this Sunday, Sept 22nd at Noon US Mountain Time (https://everytimezone.com/s/7abd4955). We’ll provide an update tomorrow to confirm the maintenance window.

Thank you to everyone who brought this to our attention and we apologize for the inconvenience.

Hop ! Après l'appel à la modération, 2 round de la maintenance de jlai.lu ! Cette fois ci, nous abordons la gestion des petites communautés tapis dans le grenier de jlai.lu.

Donc le but, et ce n'est pas une position agréable pour moi de le proposer, serait de faire le ménage.

De savoir si les modérateurs et modératrices sont présentes et ce qu'iels souhaitent faire. Nous allons relancer ces communautés si besoin d'aide. :D

Nous gardons le meilleur pour la fin : la semaine prochaine, nous ferons la promotions de vos communautés ! Et j'aimerai que nous l'organisons mensuellement. Le but serait de leur donner un second souffle ! :D

C'est Ti-par ! La suite dans les commentaires :)

tl;dr summary furry.engineer and pawb.fun will be down for several hours this evening (5 PM Mountain Time onward) as we migrate data from the cloud to local storage. We'll post updates via our announcements channel at https://t.me/pawbsocial.

In order to reduce costs and expand our storage pool, we'll be migrating data from our existing Cloudflare R2 buckets to local replicated network storage, and from Proxmox-based LXC containers to Kubernetes pods.

Currently, according to Mastodon, we're using about 1 TB of media storage, but according to Cloudflare, we're using near 6 TB. This appears to be due to Cloudflare R2's implementation of the underlying S3 protocol that Mastodon uses for cloud-based media storage, which is preventing Mastodon from properly cleaning up no longer used files.

As part of the move, we'll be creating / using new Docker-based images for Glitch-SOC (the fork of Mastodon we use) and hooking that up to a dedicated set of database nodes and replicated st

Lemmy Cafe will be having its database upgraded.

Reasons

• Pict-rs is expecting PostgreSQL 16. It's running fine now, but it might not be at some future point.
• PostgreSQL 15 has a bug that requires jit to be turned off - otherwise DB keeps consuming all the memory available on the system and then some. In the end it gets culled by the kernel. This has performance as well as reliability issues. While turning jit off has remedied the constant failure, it has also made the database a bit slower. I prefer squeezing out as much performance as possible :)

Plan

• Point nginx to the maintenance page
• Shut down PostgreSQL 15
• Run the upgrade tool
• Start up PostgreSQL 16
• Point nginx to lemmy

Expected downtime

About an hour, if things go well. More if not so.

Will try to keep the maintenance page updated.

Here's the timezone converter

Currently, we’re running the Ubiquiti Dream Machine directly as the modem via PPPoE, but there appears to be an intermittent issue with the software implementation that results in periodic downtimes of a few minutes while it reconnects.

We’re looking at switching this back out with the ISP provided router in pass through mode to negate the PPPoE connectivity drop.

We don’t expect this to take longer than 1 hour to switch over and test for reliability before bringing the services back up.

We’ll be performing this maintenance around 11 AM US Mountain Time, and will provide updates via the Telegram channel at https://t.me/pawbsocial.

We will be upgrading to Lemmy v0.18.3 on July 28 4PM UTC. During this time, this instance will be temporarily unavailable for approximately one hour as we perform the upgrade.

You can view the instance status updates here: https://status.ani.social/

Thank you!

Hi, I am going to increase the RAM assigned to the VM hosting this Lemmy instance.

The server will be offline and will be restored before 1.00 PM CEST.