Laya on CoreML: 99.5% of ops on the Apple Neural Engine
FluidInference ported Laya to CoreML with 99.5% of ops running on the ANE and benchmarked it at 3.7 ms per decision on an M5 Pro.
FluidInference ported Laya to CoreML with 99.5% of ops running on the ANE and benchmarked it at 3.7 ms per decision on an M5 Pro.
介绍比Jev快50倍,在你设备上跑的laya-mlx! 只在你的设备上占用最高1G内存 Laya是一个开源的类似于Jev的,基于文本输出概率的分类系统 我将其移植到MLX,并且做了一些性能优化! 视频中就是这个模型在我的本地M3Max上玩贪吃蛇 这个模型能够以每秒决策60次的速度玩贪吃蛇! https://t.co/31KGUiNunb
@mizorewww · Tools & apps · free, local · 60 decisions/s
@mizorewww
X post
Tools & apps
free, local
60 decisions/s
ONNX export of the Laya checkpoint so it runs anywhere onnxruntime does, including browsers and edge boxes. An fp16 variant followed a day later.
Mattepiu · Tools & apps
Mattepiu
GitHub
Tools & apps
-
-
The problem isn't the models, it's how you define the problem space, design the movements / decisions and optimize for it (batching e.g.) Jev wins for a network-based model any day. Local models tied - Laya-mlx and my custom build Jax. Look at them go.
@tomhacks · Tools & apps
@tomhacks
X post
Tools & apps
-
-
Jev vs locally running Laya-MLX and Kev-4B I built my own snake benchmark. Jev @typesafeai is running via API. The other models are small alternatives running on very little RAM on my Macbook! Jev seems to deliver the best quality no doubt! After running it for a while 0
@tobiaswup · Tools & apps
@tobiaswup
X post
Tools & apps
-
-
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference… Models: huggingface.co/FluidInference…
This is NOT Jev. Open source. Runs on your laptop. Decides in ~27 ms, about 200× faster than waiting on a hosted LLM. Here it is playing Tetris by itself 👇 brainfunctioncollapse.com/laya