使用函数式编程实现TOKENIZER - Rust编程小技巧
4:46
使用函数式编程实现TOKENIZER - Rust编程小技巧
1.3K views8 months ago
bilibiliizhangshuo
I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more.Wrote the full transformer architecture, and BPE tokenizer from scratch.The framework features:- Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x increased throughput- Automatic WebGPU fallback for non-NVIDIA devices- TypeScript API with Rust compute backend- One npm install to get started, prebuilt binaries for every platformTry out the model f
2:12
I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more.Wrote the full transformer architecture, and BPE tokenizer from scratch.The framework features:- Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x increased throughput- Automatic WebGPU fallback for non-NVIDIA devices- TypeScript API with Rust compute backend- One npm install to get started, prebuilt binaries for every platformTry out the model f
784.6K views3 weeks ago
x.comAadi Kulshrestha
How a Shell Reads Commands | Building a Tokenizer in Assembly
39:18
How a Shell Reads Commands | Building a Tokenizer in Assembly
1.8K views5 months ago
YouTubeMxy
L-3 | LLM Tokenizers Explained: BPE, SentencePiece, Pretrained vs Custom (Full Hands-On Guide)
42:08
L-3 | LLM Tokenizers Explained: BPE, SentencePiece, Pretrained vs Custom (Full Hands-On Guide)
3.2K views5 months ago
YouTubeCode With Aarohi
What is a tokenizer? 🤯 the simplest explanation you’ll find 🫡
1:28
What is a tokenizer? 🤯 the simplest explanation you’ll find 🫡
1.5K views2 months ago
YouTubesayed.developer
L-3 | Building LLM Tokenizers From Scratch (With Code!)
37:31
L-3 | Building LLM Tokenizers From Scratch (With Code!)
3.4K views5 months ago
YouTubeCode With Aarohi Hindi
Tokenizer - The Hidden Layers of #FoundryVTT 's 6th Most Popular Module
43:09
Tokenizer - The Hidden Layers of #FoundryVTT 's 6th Most Popular Module
6.5K views9 months ago
YouTubeBaileywiki
6:11
6-2 Training Your AI Tokenizer
13 views7 months ago
YouTubeVu Hung Nguyen (Hưng)
0:45
Claude Opus 4.7 update: tokenizer changes
1.1K views3 weeks ago
YouTubeSai Santosh Kumar (SSK)
33:45
L-10 | Train Domain Specific Tokenizer for LLLMs
2K views2 months ago
YouTubeCode With Aarohi
1:01
Tokenizer Tutorial: Customize VTT Tokens Like a Pro!
1.6K views9 months ago
YouTubeBaileywiki
6:13
Tokenization: The Cursed Trick that Unlocked LLMs
128 views1 month ago
YouTubeNeural Download
1:06
Train an AI model using Rust!
202.1K views3 weeks ago
YouTubeArnitly
8:58
Talifun Tokenizer - Lowest Latency / Highest Throughput
41 views2 weeks ago
YouTubeTalifun-Tokenizer
1:13:47
Duo vs 1000 Players; How we DOMINATE Official Rust...
65K views3 weeks ago
YouTubeCali
5:58
The New Best Settings For Rust 2026 (Increase FPS)
64.7K views4 months ago
YouTubeBuild Lab
1:33:56
400К СЕРЫ! СЛОМАЛИ ЭКОНОМИКУ СЕРВЕРА ЗА 24 ЧАСА. ОГРАБИЛИ ДВА КЛАНА в раст rust
82.5K views1 month ago
YouTubeGLUTTON
2:55:15
We Lived Among Enemy Pro Clans for 90 Hours in Vanilla Rust...
35.6K views3 weeks ago
YouTubeFriduwulf
17:17
The Snare - A Solo/Duo MINI TANK with INSANE Bunker
23.8K views1 month ago
YouTubeZypic
13:02
The BEST Rust Settings for FPS and PVP in 2026 (FPS Boost, Reduce Latency, Fix High Ping)
75.2K views4 months ago
YouTubeSTELIC
10:32
Token 到底是什么？—— 揭秘大模型背后的“文字压缩术”
68.6K views4 months ago
YouTube马克的技术工作坊
36:04
Build Your First Neural Network in Rust 2026 - Spam Classifier from Scratch
783 views3 months ago
YouTubeAarambh Dev Hub
0:36
Custom Parser for My Compiler in Rust
18 views8 months ago
YouTubeArtemisFx
11:08
Build Your Own Markdown Parser From Scratch (Tokenizer)
875 views10 months ago
YouTubeNetcreed
2:13:35
Let's build the GPT Tokenizer
1.1M viewsFeb 20, 2024
YouTubeAndrej Karpathy
2:18
The Best Free Rust Script // ALL GUNS // (UNDECTED 2026)
96.6K views9 months ago
YouTubeArrow Cheats
3:05
The New Best Settings For Rust 2026 (Increase FPS)
179.1K views8 months ago
YouTubeSBJake
30:42
ULTIMATE Rust Optimization (NEW UI UPDATE) | Max FPS | Best Settings
79.9K views9 months ago
YouTubeTroubleChute
4:36
L10: Wrap it with pre trained tokenizer
893 viewsDec 30, 2024
YouTubeIIT Madras - B.S. Degree Programme
1:06:11
Tutorial 02 : Create a Tokenizer from Scratch | Build an LLM From Scratch
1.1K views11 months ago
YouTubeKNOWLEDGE DOCTOR
See more