CloudTube progress update

I expected I'd write about this sooner, but I haven't until now. I've come across posts on various forums online with people talking about CloudTube and writing theories about how well it works, how well it may work in the future, and what I'm up to personally.

If there's a question you'd like to ask, I do publish my contact details on this site, so go for it! CloudTube also has a public mailing list and ticket tracker if you have a more focused question that you'd like to make public.

In the meantime, I'll deal with some popular questions here. I could write entire blog posts about each, but I'll try to keep the answers short for all of our sakes.

What will happen to old CloudTube?

It'll most likely redirect to new CloudTube once the new site implements all the important features that the old site had. You'll have the opportunity to export your data and import it into another program.

Something something rate limited by YouTube?

YouTube automatically blocks all IP addresses that send many requests (something in the range of 1000/day) and then automatically lifts the block after a few more days pass. This is because YouTube wants to stop people like me from doing this exact thing.

Obviously, after spending years in the alt-tube community, I am aware that this block exists and how it works, and I would not have started building this site if I did not have multiple plans for how to get around it.

I have written a lot more about this issue and the ways it can be dealt with. If you don't want to read that and just want a solution now, change instances in settings. Invidious and Second are supported.

Planning to develop new CloudTube more?

Yes. My classes are taking priority for the next few weeks. After that, we'll see.

How do I selfhost this?

You don't yet unless you're really good at computers and guessing. I'll write documentation for how to do it at some point.

Tor? Proxies? VPN?

They'll all be blocked eventually. Not sure about the implementation details yet, but it will happen. Select exactly one of the following options:

Why is it so much faster than old CloudTube?

Second is fast. It's also hosted on the same machine, so you don't have to wait for my Australian server to connect to some Invidious instance in Germany before the page loads.

Why is the subscriptions page so much faster than old CloudTube?

It updates in the background, so all the videos are there waiting for you. It also uses RSS feeds instead of Invidious for fetching the actual data - at the disadvantage of not having video durations available, which I think is a worthwhile sacrifice.

Will this scale?

That's a very important question! I don't know yet. But I think it will:

Most of the time serving a request is spent waiting for the Second/Invidious instance, and now you can select which instance you'd like CloudTube to use instead of being stuck on whichever one I hardcoded into old CloudTube.

A person's subscriptions are only updated in the background if they have recently logged in, so if someone quits the site forever, we won't spend time updating their subscriptions. But will they still scale if the site gets too many users? Let's crunch the numbers.

Subscriptions can be updated at around 80 channels per minute using the current approach of not sending any requests in parallel (I could totally try sending requests in parallel if I need to). From registered users on old CloudTube, there are has 15k channel subscriptions, which is 10k channels after duplicates are removed. But if I only count people who had an active login session within the last month, then there are 2.3k subscribed channels, which is 2k after removing duplicates. If I only count people who have an active login session right now, those numbers are 1.9k and 1.7k.

Let's use 2k as our number of channels to update - it's possible some people understandably left the site because it's been breaking recently. 2000 channels at 80 channels per minute is 25 minutes, meaning at the current size of old CloudTube, channels would be refreshed every 25 minutes, meaning that videos could only be missed in the feed for 25 minutes, but they could in fact possibly appear much sooner if you get lucky.

As this scales with more users and more subscriptions, the performance of the subscriptions page loading won't get worse, only the average time that it takes for a video to appear in your feed will. However, all channels in the queue will be refreshed eventually - I'd consider a queue time of 6 hours to be usable. At 6 hours to deal with the queue, the site would have 28k unique subscribed channels from active users - and that's without parallelism.

If this somehow still manages to go to shit and some metric performance becomes unacceptable, you'll be pleased to hear that it'll be pretty easy for people to host CloudTube and Second themselves, which means that more public instances should spring up for people to use. It also means that hosters don't have to deal with all the other personal stuff on my site that was all mixed in with old CloudTube.

Why did I design old CloudTube that way in the first place, as just another part of my old site, you may ask? It's because I didn't expect it to get even slightly popular. You'll be pleased to hear that new CloudTube will be its own standalone thing, exactly as it's shown on https://tube.cadence.moe .

Some feature is missing!

Go to the ticket tracker and put it in there so I don't forget to add it.

Something doesn't work!

Yeah, I'm not surprised. Put it in the ticket tracker so that I know about it.

You're cool!

Thanks, I try. Being cool doesn't help me do this stuff better/faster/sustainable-er though. If you like my stuff then please give me money - I spend literally hundreds of hours of code for free, and host it for free, and release the code online for free, and let other people use it for free. (Great segue, Cadence!)

We can make a table of my money situation, if you like:

Things I do and give away for freeThings I get
make my websitetens of thousands of dollars of student loan debt
write on this blogmisgendered online
understand how YouTube worksabout two donations per year
make old CloudTube
make new CloudTube
make Bibliogram
make Amanda
make a bunch more stuff

Someone who is good at the economy please help me

— Cadence

← Previous: My experience with the Workman layout