Skip Navigation
Dubvee Meta

Announcements and meta discussions about the DubVee instance.

Members
98
Posts
36
Active Today
1
Created
2 yr. ago
  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    I've reached the end of my rope with regard to cleaning up spam originating from lemdro.id.

    I'd been debating this for a while, but after running some reports from the database this morning, 33% 35% of all total Lemdroid accounts are spam with that number jumping to 67% 75% for accounts registered year-to-date (those numbers are possibly low since that is only for identified spam that this instance knows about). I don't defederate lightly from an otherwise good instance, but seeing the actual percentage of spam users there plus the burden having to keep cleaning up has pushed me over the threshold.

    Looking at the communities subscribed here, there are alternatives to some of the Lemdroid communities:

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Upgraded to 0.19.9

    As always I would like to thank the Lemmy devs for their wonderful work of making super robust database migration scripts that don't cryptically fail /s

    FYI: I'm probably looking at migrating this instance to Piefed in the near future (in lieu of Lemmy 1.0). That may be an issue if you use 3rd party Lemmy apps, and I'm hoping to come up with a solution for a direct migration (if such a solution does not already exist).

    The Piefed dev said their API is pretty similar, so I may be able to add a translation layer. I dunno if that's possible, but I will be looking into it. Just wanted to give a heads up on that. Bottom line is I'm fed up and just about done with Lemmy as a platform now that there are viable alternatives.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    They're not enforcing their own rules as of late and letting unsourced screenshots, social media, and other non-article opinion drivel remain up. I'm getting tired of modding it locally so that the content complies with our misinformation policy.

    Until they get their shit together, please use:

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Expect downtime: Mother nature's pissed off

    Update 10:25 PM: Back online. It's going to take a bit for the instance to catch up from being down for almost 12 hours, but more instances seem to be reporting in /catching up except Lemmy World. Federation stats say next retry is 2025-02-17 06:45 AM. Not sure if there's anything I can do to kick that into gear sooner, and once it does start catching up, it'll probably take ~8 hours to get back in sync.

    Apologies for the massive downtime (10 hours is "massive" by my standards). Got 2 more inches of rain on top of a week of rain which was on top of 3 inches of rain last week. Mudslides, rockslides, and flooding incidents were all over the place today plus some trees that uprooted. Not sure which of those was responsible for my outage, but could have been any or all.

    On-prem, thanks to a clogged storm drain, my generator ended up half-submerged in water. Haven't had a chance to inspect it, but it's probably going to need repairs and is down for the count until then. :(

    In

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    It's my birthday 🎉🎉🎉

    I'm celebrating by not celebrating and just getting drunk with some friends which is different from celebrating because we all have to work in the morning.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Emergency Maintenance Today

    SSD in the load balancer is showing signs of death, and I'm going to have to deal with that today.

    I've spun up a temporary, virtual LB on another host and am re-routing traffic to that, so hopefully there won't be any noticeable downtime. That said, things could go awry, so just throwing this notice out as a CYA in case.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Why DubVee and How We're Different

    Preface

    I've run DubVee for about a year-and-a-half as of this writing. Until now, I've not really had much of a mission statement beyond being an on-ramp to the Fediverse and trying to keep things civil and non-toxic from this instance's point of view.

    Recent and semi-recent events have revealed an ugly truth about many of the users in the Fediverse (and social media in general), so I've put together an actual mission statement in hopes of addressing that. It's also my last-ditch effort to not shut down this project out of disgust.

    I want DubVee to be an example of what the Fediverse can be when it's not overrun with political idealogies masquerading as people, extremists, lynch mobs, and people too toxic to be anywhere else. I know for a fact that there are plenty of users that don't fit that criteria, and I dare say they are the majority. However, the above-mentioned groups are definitely drowning them out and giving Lemmy (and the Fediverse) a bad image in the process.

    To

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Reminder: Any calls for, celebration of, or otherwise praising acts of violence or those committing it, even if you dislike the target, will not be tolerated.

    News flash: Bad things can happen to people you think are bad without frothing at the mouth and turning into a lynch mob.

    DubVee will not tolerate any form of extremism, vigilantism, mob justice, celebration of violence, etc. Please be sure to report any instances if you encounter them.

    -- Pat

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Brief downtime this evening. Shooting for 6pm EST

    Two of the three new UPSs arrived, and I'm going to go ahead and swap them in versus waiting for the third one. I'll just have to deal with a little extra planned downtime later to install the 3rd one since there's too much potential for unplanned downtime with the two faulty ones in place now.

    Shooting for 6 pm or very close to that. I'm going to cable-manage a bit, so planning for between 10 and 30 minutes of downtime.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Potential service instabilities for the next 18-24 Hours

    Got some wicked winds roaring through here. Power has been flashing like crazy, and surprise, both UPS's are crapping out when I need them (even the one I replaced a few months ago 😠). They're both tripping off after 1-2 seconds.

    I ordered three new UPS units Monday, but they're not scheduled to arrive until tomorrow evening. I will probably just go ahead and install them as soon as they are delivered without much warning since the current ones are in such poor shape.

    I guess Tripp Lite is no longer a decent UPS brand in 2024?

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Fiber cut in my area. FML.

    Fiber's shitting the bed, apparently a cut. We're on backup WAN. ETA from ISP is 5am tomorrow

    Upgraded the backup connection last week, so will leave pict-rs enabled for now and see how things go.


    Update: Fiber came back online around 4:30 this morning and the auto fallback seems to have worked. Yay.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Fiber provider is absolutely shitting the bed today

    This is the second time today I've had to cut over to the backup WAN (which is much slower). Fiber provider is royally screwing the pooch today, and I currently have no ETA. The only silver lining is it's a widespread issue, so hopefully they're inclined to address the problem in a timely manner.

    To relieve congestion on the backup connection, I've temporarily disabled external access to pictrs. All requests (GET or POST) to /pictrs will return a 404 response until the main connection is back online.

    Sorry for the inconvenience. Please direct all rage at the shit fiber provider I foolishly "upgraded" to.

    Status as of 5:17PM:

    Service Status Notes
    Lemmy ActivityPub Up (Degraded) Running on backup WAN
    Lemmy API Up (Degraded) Backup WAN
    Pict-rs Down Intentionally scuttled to save bandwidth
    Tesseract Media Proxy Down Intentionally scuttled
    Matrix Server Up (Degraded) Backup WAN
    Matrix Web Client Up (Degraded) Backup WAN
    DNS
  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Fuuuuuuck Appalachian Power (aka Sorry for the Outage)

    Appalachian Power is the absolute worst about giving notifications for scheduled outages. The. Fucking. Worst.

    I got an email at at 8:05 PM saying there would be a planned, scheduled outage at 8:00 PM and then 45 seconds later was plunged into darkness. Didn't bother to switch on the generator :sigh: and just decided to sit on the patio and get drunk in the waning daylight.

    This is not the first time they've done that, and it probably won't be the last. Guess my next purchase is an automatic transfer switch.

    Did I say "fuck Appalachian Power"? Because Fuck. Appalachian. Power.

    Sorry about that. lol

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    DubVee's Future, Lemmy's Extremism Problem, and Our Response to That

    Let me start off by saying, in my official admin capacity, that I really don't know if I want to be here anymore (see the problem outlined below). That has implications beyond me just deleting my account and going outside since I would not allow the instance to run unattended. I'm also unsure if my backup admin would have any interest in taking over full time.

    If it does come to that, there will be a subsequent announcement and either a transition plan or a sunset plan.

    That is still up in the air, but for now....

    The Problem

    I just wanted to take a moment here to address Lemmy's extremism problem / lynch-mob mentality and how DubVee is responding to that.

    Last month, a new site rule was added expressly forbidding any form of extremism with regards to violence. Every rule is there for a reason, and this was added as a direct response to the increasingly violent rhetoric I've been seeing from users.

    In full:

    I don't care which end of the spectrum you're on. Any post/

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Planned Maintenance This Weekend and Next Week

    This Weekend

    There will be some periodic downtime this weekend as I do some physical cleanup / rearrangement to my equipment. I'll try to keep these to a minimum, but at some point, everything is going to need to go offline for a bit in order to re-cable the UPS and power connections.

    Plan is Saturday afternoon between 1 and 4 pm. Hopefully any interruptions will be brief.

    Next Week

    I'm switching to a new fiber provider next week (FTTH vs current FTTN), so there will be a brief disruption as I cut over to that. That downtime should only be a matter of seconds since both connections will be active, but just in case it turns into a whole thing, I'm mentioning it now. The new fiber is supposed to be installed Tuesday afternoon, and I plan to do the cutover around 7PM local time.

    Next Few Months

    I'll be expanding the storage server in the coming weeks, but that should not have any immediate impact on DubVee's normal operation.

    With these, I hope to provide a better

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Upgraded Frontend to Tesseract 1.4.0 Beta

    I've been working on the latest release of Tesseract for the last few weeks, and I'm happy to announce the first beta release of the upcoming 1.4.0 series.

    I daily drive my dev version throughout the development process, so all the bugs I've encountered/introduced should be fixed. The beta test on the main instance is more of a formality and final shakedown. That said, please report any bugs you encounter either through Github or by describing them in a comment here.

    1.4.0 is a significant update as it drops all backwards compatibility with the 0.18.x API and introduces several new features. It's also had some additional polish since 1.3.0 and added things that have been requested for quite some time (pasting images into post/comments, etc).

    As always: thanks in advance for being my beta testers. Appreciate it!

    Change log for 1.4.0 if you want to see what's new: https://github.com/asimons04/tesseract/blob/1.4.0/ChangeLog.md

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Inbound federation delays from Lemmy World (Again). Workaround is in place for now.

    If you've noticed content from Lemmy World lagging by a few hours the past few days, it's not just you.

    Long story short, it's a problem with how Lemmy sends activities, and it's heavily impacted by latency between sending and receiving server and creates a fixed upper limit on the number of activities per second that can be sent. Lemmy World is hosted in Finland, and DubVee on the US east coast. There's only so much I can do to work around the inherent latency of a trans-Atlantic link.

    We're not alone in this. Some instances, particularly in Australia, New Zealand, and US west coast, are impacted more dramatically.

    While there's always some federation delay/backlog, it only escalates to noticeable levels every so often. Not sure why, but I've gone over our infrastructure top to bottom several times and cannot find any reason, on this end, for these events. The graph in the post shows the number of activities Dubvee is lagging behind Lemmy World over the last 30 days.

    Nor

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Upgraded backend to 0.19.3

    Finally got around to upgrading to 0.19.3.

    Upgrade did not go smoothly due to multiple DB migration step failures and less than helpful logging from Lemmy during the DB migrations. (surprised Pikachu).

    After digging through to figure out what it was complaining about, the upgrade was finally able to be completed.

    You will likely need to log in again (at least I did).

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Migrating Pict-rs to Postgres. Images will be unavailable for a bit

    I'm in the process of migrating the pict-rs database to Postgres. Pict-rs has to be offline during this process, so images here will appear to be broken for a bit. You also won't be able to upload any images for the duration of this migration.

    Estimated time is about 20-30 minutes.

  • Dubvee Meta @dubvee.org
    Admiral Patrick @dubvee.org

    Frontend Cleanup: Let me know if you're using old.dubvee or alex.dubvee

    Currently, we host 4 UIs:

    Due to all the bot crawler traffic that slips through, I'm having a hard time distinguishing actual usage from bot traffic. That said, it does seem like Mlmym and Alexandrite are rarely used by actual people (there's a lot of bot traffic to filter out that spoofs its user agent, so I could easily be wrong here).

    Update: Mlmym does all the Lemmy API calls server-side, so all of the actual usage traffic was actually originating from my server (which I was filtering out).

    I'm looking to decommission Mlmym (the old Reddit style) and Alexandrite.

    I just updated both of those to the latest, so apologies for the brief disruption if you were using them, but ultimately I'd like to trim down the selection to just the ones being acti