Trained a 117M parameters Silia model on an H100 in 5 hours.
This covers a coding tool or code-capability update — useful for developers assessing workflow changes and reusable value.
A 117M parameter Silia model was trained on an H100 in 5 hours using the codelion/synth-100M dataset.…
About a month ago I posted my very first paper about my custom Silia architecture here https://www.reddit.com/r/LocalLLaMA/s/J19Qi4NXeJ
With the help of Ok-Internal9317 who decided to sponsor the paper with compute I was able to train a 117M parameters model.
You can checkout the model here
Hugging Face
https://huggingface.co/Srijan-Srivastava/Strawberry-s1
GitHub
https://github.com/SrijanSriv211/Silia/
How to Generate?
Example prompt:
Which animal has more poison - the salamander that sticks out its bone or the frog with the sharp head thing, and how do they both make their enemies hurt?
Use uv for inferencing. Install torch, numpy, regex and colorama.
uv run inference.py -i 117M_fp32/final.bin -e cl16k.bin -T "Which animal has more poison - the salamander that sticks out its bone or the frog with the sharp head thing, and how do they both make their enemies hurt?"
Generated output: ``` I dont understand why they all work together.
1. Query Decomposition
"pouring your animal's survival" → food safety concern "all the animals" → dual danger threshold "potential danger" → threshold question, not just threshold
● High confidence: food safety, environmental impact, food safety
User seeking: food safety considerations, policy implications, environmental interactions
2. Domain Contextualization
Food safety (food safety): food safety, food safety, health needs "all the animals" → food safety threshold, environmental impact "all the animals" → food safety threshold, food safety concerns "did it work together" → safety safety requirements, environmental conditions
Key domains needed: - food safety physiology (flight, feeding, food safety) - environmental risk stratification - environmental factors - environmental interactions
3. Information State Assessment
● High confidence ```
Silia research paper
https://huggingface.co/Srijan-Srivastava/Strawberry-s1/blob/main/Silia%3A%20Tiny%20Scale%20Is%20All%20I%20Can%20Spare%20To%20Play%20With%20Transformer.pdf