All right, the server has been updated to the latest stable version.
I initially intended to leave it running for a few hours to make sure everything is stable, but somewhere along the server reboots, the bot got activated once more and errr, well… it’s been posting happily for some time now.
I guess that answers my question whether or not it’s still compatible with the new server version.
Dropping by for another update:
There are still some updates and migrations left to do, but things are looking up, I think the worst is behind us.
Having said that, I’ll be taking the server offline now for the rest of the updates - hope this comment was properly propagated :D
Good news everyone, the bucket migration has finished! At first glance everything seems to be in working order.
To make the most out of the downtime, I’ll continue with an attempt at upgrading to a more recent Lemmy version this weekend. So there might be a bit more downtime, but then we should be good for a while.
Small update
So it’s been over 4 days of file transferring. There are about 13.6 million files in the source buckets, and yesterday we passed 14 million file checks.
So I’m guessing that means it needs to do multiple checks per file. I’m hoping it’s 2 checks per file, meaning the process is over half way.
The good news that most of the data has been transferred over (source shows 727GB used, destination shows 713 GB).
The bad news is that due to how the files are structured, it’s incredibly hard for the sync process to estimate which files it still has to move over, so it does need to check them all one-by-one.
Ironically, me posting this image above would mean there’s 1 file more that has to be synced over. But I’m willing to let it be a casualty of this war ;)
It does not, since it’s not a bridge, but a one-way syndication.
I’ll leave that to the reddit moderators and community to decide. There are thresholds in place to ensure they have time to filter out illegal material before it gets crossposted.
And again, so far no original poster has reported any posts on these bases. But I have no qualms with taking it down if they do.
It’s noble of you to want to protect those people, but the moment they post content to reddit, they have given reddit the right to duplicate and redistribute their content as it sees fit. So the way I see it, that ship has sailed, and I’m pretty sure most creators are aware of this.
Having said that, if someone ever reports they want to take their content offline, I’ll gladly do so. So far, that hasn’t happened yet though.
Edited to add: the lemmit service doesn’t even copy most content, only relatively small (animated) images. On account of storage (and processing) larger files costs money. So most content is hotlinked. That means that when the original is removed from Reddit, it’s removed from lemmy as well.
not literally, they wouldn’t grow
Today I learned.
I might have been inclined to agree with you, if the posts themselves didn’t include all the attribution and other data that is on the reddit posts themselves.
How do you feel this robs from content creators, rather than giving them more exposure? It’s not redistributing any content that’s behind a paywall, for example.
I’ll consider your opinion.
Looks like it’s working. Time for a beer!
Ah, I guess I must have overlooked that part. There are several reasons for not wanting to allow signups.
One is quite simple, cost. Right now this is running on a small, single core instance. It often stutters (especially when handling video updates), and that is not an issue, since that just means it’s going to take small while before updates are sent out. But you wouldn’t want to have that delay for actual users. Right now the costs are quite manageable, if I have to scale up in order to provide a fluent experience for its users, not so much.
Most of the other reasons come down to the responsibility of having to provide a home to any outside users that sign up. I don’t have the interest or time to maintain a community of people, nor to guarantee the uptime that such a server would require. It also wouldn’t work. The largest Lemmy instance in existence, lemmy.world, has defederated from this instance. So any users that sign up here, would be devoid from content on there. And as you said, any other instance can decide to do so at any time (in fact, I very much suggest they do so in the FAQ).
I could go on, but I think you get my drift.
Thanks for your feedback. Let me assure you that I’m actually a giant asshole in real life ;) I do want to respectfully disagree with some of the points you make though.
If no bots cross posted things from reddit, actual users would post them, they would get updoots, feel great about themselves, and search out more content.
I don’t think that’s the case. There is no reason for people to not make a post, just because it already exists on Lemmit. In fact, I feel the opposite is the case: it’s making it easier to do so by cross-posting. People scour Lemmit news communities for potential useful links, and easily send them to the relevant “native” Lemmy communities.
Cross posting bots implies that lemmy is some kind of second class reddit. As though reddit is where the content is but we’re hanging out here in protest. Lemmy’s own culture and community needs to coalesce in it’s own time.
I would challenge you to really carefully consider what the objectives of lemmit.online are - they seem to have changed since your original “about” post (now removed ?)
I kinda get what you’re saying with this, but I don’t agree. Reddit and Lemmy are roughly the same product - communities where texts/links can be posted and discussed. The simple fact is that Lemmy is nowhere near the scale of Reddit, and so there is less users and less content to go around. That doesn’t make one better than the other - one is more rich in content and the other more tightly knit. Some of the communities rely very much on discussions, others less so. I had always intended Lemmit to boost the second group of content/communities. Communities centered around memes/jokes/nsfw content/etc - I value those because I click on the link, read the article, and move on. I still stand by that design choice, it’s just that it is now enforced - communities like AskReddit/AmITheAsshole/AskBla are no longer archived, and requests for new communities need to be manually approved.
Not sure which About post you mean, but the most info is still here in the FAQ: https://lemmit.online/post/14692
Yes, users can ban bots - but we’ve all heard of the 90 / 9 / 1 rule - 90% of the people that will visit Lemmy will not log in and therefore will not curate their feeds in that way.
Instances are reluctant to defederate from lemmit.online because doing so has the whiff of censorship - even though there’s no actual content here to censor.
I’m not convinced of this. The largest instance, lemmy.world (where most of the “inactive” users will end up) has done this, so that takes care of them. As for the latter - I explicitly give permission to instance owners to ban the bot or defederate in the FAQ, so I don’t think the defederation stigma makes any sense here.
Heya,
I still need to create some tools to make to easily add new subreddits to the bot. I’ll probably get around to that this weekend, and then I’ll add /r/theyknew and notify you. As far as I’m concerned, it’s a great contender for synchronisation/archiving.
Can’t blame you for that. Personally, I still think it excels at content where communication with OP is irrelevant, like [email protected], [email protected] or [email protected]. And by far best example of this, if you look at the subscriber count, is nsfw content.
Nope. That would be very hard to implement, and probably very confusing and disliked by other lemmy users.
I don’t know how the karma thresholds work behind the scenes, but might I suggest for the bot to do a “top for” sort instead? Like it will only repost top content for the past 6 hours only. This will also help get more quality content as well and avoid reposting low effort/quality posts.
This is effectively already kinda how it works. For each subreddit it periodically (anywhere between every 30 minutes to every 12 hours, based on subscriber count and posts per day) requests the “hot” content feed. It then checks each post if it has at least 20 upvotes, and a 80% upvote to downvote ratio. Those numbers are configurable, but that’s what they’re currently set to - I believe they’re a good mix between filtering out the complete garbage while still making sure it doesn’t miss good content is.
Good. :)