From what I've seen, it's definitely worth quantizing. I've used llama 3 8B (fp16) and llama 3 70B (q2_XS). The 70B version was way better, even with this quantization and it fits perfectly in 24 GB of VRAM. There's also this comparison showing the quantization option and their benchmark scores:
To run this particular model though, you would need about 45GB of RAM just for the q2_K quant according to Ollama. I think I could run this with my GPU and offload the rest of the layers to the CPU, but the performance wouldn't be that great(e.g. less than 1 t/s).
Just one day after Meta released their new frontier models, Mistral AI surprised us with a new model, Mistral Large 2.
It's quite a big one with 123B parameters, so I'm not sure if I would be able to run it at all. However, based on their numbers, it seems to come close to GPT-4o. They claim to be on par with GPT-4o, Claude 3 Opus, and the fresh Llama 3 405B regarding coding related tasks.
It's multilingual, and from what they said in their blog post, it was trained on a large coding data set as well covering 80+ programming languages. They also claim that it is "trained to acknowledge when it cannot find solutions or does not have sufficient information to provide a confident answer"
I also really like that model and their fine-tunes. If licensing is a concern, it's definitely a great choice.
Mistral also has a new model, Mistral Nemo. I haven't tried it myself, but I heard it's quite good. It's also licensed under Apache 2.0 as far as I know.
Just one day after Meta released their new frontier models, Mistral AI surprised us with a new model, Mistral Large 2.
It's quite a big one with 123B parameters, so I'm not sure if I would be able to run it at all. However, based on their numbers, it seems to come close to GPT-4o. They claim to be on par with GPT-4o, Claude 3 Opus, and the fresh Llama 3 405B regarding coding related tasks.
It's multilingual, and from what they said in their blog post, it was trained on a large coding data set as well covering 80+ programming languages. They also claim that it is "trained to acknowledge when it cannot find solutions or does not have sufficient information to provide a confident answer"
On the licensing side, it's free for research and non-commercial applications, but you have to pay them for commercial us
Bringing open intelligence to all, our latest models expand context length, add support across eight languages, and include Meta Llama 3.1 405B— the first frontier-level open source AI model.
Link Actions
Meta has released llama 3.1. It seems to be a significant improvement to an already quite good model. It is now multilingual, has a 128k context window, has some sort of tool chaining support and, overall, performs better on benchmarks than its predecessor.
With this new version, they also released their 405B parameter version, along with the updated 70B and 8B versions.
I've been using the 3.0 version and was already satisfied, so I'm excited to try this.
I haven't tested it extensively, but open webui also has RAG functionality (chat with documents).
The UI it self is also kinda cool and it has other useful features like commands (for common prompts) and searching for stuff online (e.g. with searx). It works quite well with Ollama.
Currently, I only have a free account there. I tried Hydroxide first, and I had no problem logging in. I was also able to fetch some emails. I will try hydroxide-push as well later.
I haven't heard of Hydroxide before; thank you for highlighting it! Just one question: Does it also require a premium account like the official bridge, or is it also available for free accounts?
This reminds me of one of my older projects. I wanted to learn more about network communications, so I started working on a simple P2P chat app. It wasn't anything fancy, but I really enjoyed working on it. One challenge I faced was that, at the time, I didn't know how to listen for user input while handling network communication simultaneously. So, after I had managed to get multiple TCP sockets working on one thread, I thought, why not open another socket for HTTP communication? That way, I could incorporate a fancy web UI instead of just a CLI interface.
So, I wrote a simple HTTP server, which, in hindsight, might not have been necessary.
An Lemmy client for Android, forked from the Infinity for Reddit project.
Link Actions
Hello everyone!
I have some good news! Eternity has finally been added to the main F-Droid repo. I've managed to get reproducible builds working, so this version is the same as the one on Codeberg (but verified by F-droid).
I discovered a channel called 'Just Alex' awhile ago, and I've binge-watched most of his videos. He makes videos about a variety of activities he engages in. For example, two years ago, he started beekeeping as a hobby. In these videos, he shows the progress he has made since then, taking care of his bees and harvesting their honey.
He also has videos on collecting mushrooms, growing crops, and traveling around Europe.
What I like about him is how calming his videos are and the amount of effort he puts into making them interesting, without resorting to clickbait content.
🚀 **Changelog** 🚀
🔧 **Bug Fixes**:
- **Community Update**:
- Properly updates communities linked to different accounts.
- Improved updating of subscribed communities after subscription.
- **Login & Authentication**:
- Always includes access token in the retrofit holder, enhancing lo...
Link Actions
Hello everyone!
It's been some time since our last update. As the version number indicates, this is primarily a bugfix update; hence, it doesn’t bring the "big new feature" I had hoped to introduce. The most important change here is that this version of Eternity will be compatible with the upcoming Lemmy version 0.19.
It's worth noting that upon upgrading to this version, you may need to log out and then log back into your account, based on my experience.
For the next release, I want to address more of the common issues found in our bug tracker. So, there may be another minor update shortly, but I can't make any promises at the moment. I've also made some progress with multi-community support and I plan to introduce modding tools as well.
In the meantime, thank you so much for your continued support.
I don't think it's necessarily a third-party reseller thing. I bought tickets a few days ago literally from their app, only to get the same email that said I needed to 'verify my identity' because I bought the tickets for an 'unauthorized third-party reseller'.
I'm excited to bring some news to our wonderful community today. After an engaging voting process on the voting thread, we as a community have decided to change the name from Infinity for Lemmy to Eternity. 🎉 I am grateful to everyone who participated and voiced their opinions. It was great to see so much involvement!
While [email protected] will be our new gathering space, I'd like to let you know that the original [email protected] won't be closing its doors immediately. This way, everyone has enough time to transition comfortably and get accustomed to our new home.
Lastly, remember, while names may change, our spirit and camaraderie remain eternal. I cherish our roots with "Infinity for Reddit" and am excited about the journey ahead in "Eternity". Let's collaborate, share, and continue building on this vibrant community.
A Lemmy client forked from the Infinity for Reddit project
Link Actions
Hello everyone,
I'm not sure this is the right community for this, but I want to announce my new project: Infinity for Lemmy.
Basically, this is a fork of the Infinity for Reddit application, modified to get it to work with Lemmy.
But I must remind you to temper your expectations as this project is still in its infancy. It has some basic features already in place, but there’s a lot more to be accomplished.
Welcome to my personal lemmy instance! This is a place where I share my thoughts, opinions, and interests with the world. Lemmy is a federated, open-source, and privacy-friendly alternative to reddit. You can learn more about it here: https://join-lemmy.org/
I created this instance mainly for myself, but I’m open to accepting requests from other people who want to join, especially if they are from Hungary or speak Hungarian. If you are interested, please send me a message with a brief introduction and why you want to join. I will review your request and approve it if I think you are a good fit.
Please note that this instance has some rules that you need to follow:
Be respectful and civil to other users. No hate speech, harassment, trolling, or spamming.
Stay on topic and relevant to the community. No off-topic or low-effort posts or comments.
Follow the terms of service and code of conduct of lemmy and the fediverse. No illegal or harmful content.