This framing is already outdated.

It assumes that the problem is to outperform existing architectures under the same design constraints: similar toolchains, similar abstractions, similar iteration cycles. It assumes the bottleneck is architecture, ISA or pure execution.

The old chip design stack was built around human scarcity and walled gardens. RTL, compilers, EDA flows, licensed IP, NDAs, verification boundaries, and organizational handoffs all made sense when the central problem was helping humans divide a system too complex to hold in one’s mind.

Even with AI agents applied to chip design everybody rushed to make the old stack faster, write verification faster, help with synthesis. All these efforts are empowering the same teams with the new tools, but at the end of the day Conway’s law still means you “ship the [old] org chart”. Same happened with SW at first with co-pilots that were effectively autocomplete on steroids, before agentic systems entered the scene.

The next frontier will be defined by how many architectures can be explored, tested, refactored, and specialized by small teams using AI as a true collaborator. But how to open the chip design space to more people than just a handful of well-funded chip designers? What is missing?

The answer is not a better RTL generator.

Agents change the search space

Agents do not need to respect the boundaries humans created between software and hardware, they don’t need PyTorch, CUDA, PTX or similar abstraction layers.

If you can vibe-code directly in assembly, you don’t need a compiler.
If you can reliably vibe-code RTL, you don’t need the layers above it.
If the design process itself becomes programmable, even RTL starts to look like an unnecessary intermediate representation. We are not suggesting vibe coding GDS just yet, but you get the idea.

Should you be asking AI to create chip design from scratch? Not really – it is an infinite search space. What agents can do is reason about model structure, data movement, memory layout, compute topology, scheduling, microarchitecture, and RTL as coupled choices. This turns hardware design into an end-to-end optimization problem: given a workload and constraints - find the best implementation across the software-hardware boundary. It’s still the human’s job to choose the constraints and the optimization function, otherwise the search will not converge, and you’ll get a hallucinated result.

Collapsing the stack

Intent, optimization function and constraints – how do we start?

Luckily, we are not the first ones to start asking these questions.

One approach that works well is narrowing the goal. And the most prized goal right now is running AI models at the top speed ASICs are capable of. Models are defined by compute graphs and those are not Turing complete programs. If you fix the model, you can take the model graph and lower it directly into hardware implementations, bypassing traditional layers – etch the model into the chip. That’s what Taalas did compiling models directly into Verilog, showing 50x improvement vs Nvidia.

This approach took away the notion that abstraction stack is sacred.

But it only makes sense economically if that one model you lower is extremely popular and that constitutes a big market segment. For fast evolving markets or proprietary models, it doesn’t work that well.

If you want some level of reusability of your ASIC, the constraint space is not as tightly defined, so you need to introduce some other guardrails to make the search reasonable. E.g. give agents a structure, or a substrate to map models to. And in an interesting experiment in the University of Toronto (Talos V2), team of researchers and agents started with transformer model structure not a single model and ended up discovering a need for a substrate to map it to - some kind of reusable tile. Spoiler alert: they chose systolic arrays. More flexible yet not an architecture from scratch.

With the explosion of Physical AI use cases, the requirements for the diverse collection of chips are back in the game, too – it’s not just one homogeneous data center environment we are talking about – size, power, latency and other constraints, but new models and model architectures create pressure that old school chip vendors can’t sustain.

Now another question, what if we can give agents a real flexible substrate that can be morphed into many things?

Open Substrate

The substrate has to be real. It must come from hardware that has survived contact with implementation and usage, not just from academic exercises that have never been taped out.

The substrate has to be open. No IP licenses, no NDAs, no worries about foundational LLMs CLAs - agents are welcomed.

The substrate has to have tests, simulation, co-simulation, synthesis paths, examples, documentation, and clear contribution flows.

Luckily now there is a substrate like that.

And it lives under the name CORE-ET in the OpenHW Group of the Eclipse Foundation.

CORE-ET is not a white paper about future chip design. The Erbium branch is the exact design that we have taped out and expect back from TSMC in a few months. It’s also a piece of a bigger design that has been taped out 5 years ago and still stands as a useful and very energy efficient chip sporting 1088 minion cores.

How sophisticated are the Minion cores? Here comes the geeky part. Minion is a tightly optimized, dual-threaded RISC-V core: small enough to replicate aggressively, predictable enough to compose into larger fabrics, and flexible enough to sit at the center of many different architectures. It has an optional vector/graphics coprocessor for dense data-parallel work.

But the real unit of design is not the core. Eight Minions are packaged together into an atomic compute tile: efficient, coordinated, and shaped for systolic-array-like fabrics. If you want a hint about the design philosophy, remember that the T in ET stands for Transputer.

That is what makes CORE-ET interesting as substrate. It is a reusable unit of compute: compact enough for experimentation, powerful enough to matter, and structured enough for agents to reason about.

The gory microarchitectural details are here.

But! It’s not just RTL, it comes with real chips, programmer’s manual, simulator, runtime, optimized kernels, even a bare metal Go implementation. It can exist in the old school paradigm. 1 tile delivering roughly 1 TOPS at 0.5W.

But for us this RTL is not the final product. It is the source material. It contains real implementation knowledge: the decisions, scars, constraints, and structures that only exist when a design has been pushed toward silicon.

And the things that we were able to do with it (with the help of agents) are the beginning of a bright future where the chip design gets unlocked for millions of developers.

New Economics

Once design is grounded in a composable substrate, the entire cost structure shifts.

Iteration becomes cheap.
Exploration becomes parallelizable.
Specialization becomes viable at a much smaller scale.

The limiting factor is no longer the cost of committing to a design, but the ability to search through possibilities efficiently.

Even better the composability finally gives a boost to chiplet design. Tape out building blocks. Recompose them for different use cases. Increase the agentic search space to multiple composed blocks. Get custom chips without custom tape outs.

The difference is not incremental. It is structural

It’s the difference that can serve the demand of a tape out every 9 months even if you are not Tesla.

What’s next

CORE-ET is not the only thing we have. We have a treasure trove of IP developed over the course of 10 years by a talented team of engineers in the old paradigm. Like any other chip company on the mission to fight Nvidia they have developed a lot of things (even the RBOX!), and we aim to keep open sourcing them with the same purpose of getting more substrate for agents.

Luckily the substrate is so flexible that with agentic help you can turn it almost into anything – from systolic array architecture to Turing complete coherent blocks, from 8 core to 4000+ cores, we even have big out-of-order cores that are not so easy for agents to design right now but maybe with the right substrate?

We also are open source people, so we couldn’t shy away from open tools. And guess what? With small enough designs you not only can put them in FPGAs by tool calling Yosys in the matter of hours, but you give the open source SkyWater PDK to the agents and go through place and route with OpenROAD and you get yourself a Tiny Tapeout. Not just one chip-per-student anymore – one chip per Claude session!

Memory is next. It’s already emerging as the next bottleneck as both Taalas and Talos discovered. How to store the weights throughout the system, how they move, how they are reused, and whether they should move at all is defining the next phase of substrate design.

That deserves its own post.

For now, the point is simple: the old race was to build one giant architecture to serve them all.

The new race is to make the next thousand chips.

Join discussions on Discord

Ainekko purchased Esperanto’s IP and is developing in the public - the chip was and still is a good idea; we believe that there’s a place for a open computing platform based on said IP, providing beyond what GPUs can offer architecturally. ET-SoC-1 is the beginning. We are taping out our test chip. Until the package from TSMC shows up, ET-SoC-1 is what I’m working on.

That’s my elevator pitch as an engineer working at Nekko.

As Nekko develops in the public: I can share almost everything we do. Ironically this is the best devrel possible. Show, not only tell. I am partially done with basic kernel work in our LLM inference stack and starting to look at the processor as a proper systolic array and see what is possible. Looking at Esperanto’s code. There’s little use of the fact that the processor is a grid of cores. Either Esperanto figured out that it is not worth the effort or they are unable to make use of that fact. I don’t know which. And I care about my performance running FlashAttention and matrix multiplication on llama.cpp. Yet efforts continuously end up with “the bandwidth doesn’t make sense and I am optimizing blind”. To understand what we are working with, this is the rough NoC layout:

And so experiments we do. First, what’s the bandwidth across shires? To be specific, what if we transfer a large-ish chunk (960B, to avoid aliasing in L2) across pairs of shires via TensorLoadL2SCP (think: hardware DMA against other shire’s scratchpad)?

Despite potential shires being yielded for.. yield reasons. The internal logical layout seems to be clean. The bandwidth shows a clear-ish 8x4 pattern in bandwidth. Matching the SoC-1 datasheet’s core layout with shire id incrementing in the y direction. Bandwidth reduces per hop away, but I assume that’s more latency than throughput. And the NoC is symmetric, there is no difference in bandwidth between sending and receiving.

Question: does the shire id map to the physical location? ... We see bands so that has to mean NoC hop. But it is scrambled and not the simple pattern of physical location = shire id. Running simulated annealing we can figure the topology:

The next natural question: How does congestion work on this NoC? Does it work like some chips where each link between nodes can congest? Is the NoC half or full duplex? Does the NoC have directionality?

Measurement results say the NoC is bidirectional and/or the bandwidth is so high that the amount of traffic can be sustained without congestion. I cannot tell if the NoC is sized to match the rest of the chip (maybe.. but questionable choice as that means most usage will underutilize the bandwidth and waste of chip power and area) or the simple pattern is too trivial and Esperanto has designed to solve the specific case. I can’t tell from numerical results. Some further pattern testing shows the same thing:

Scenarios:

• disjoint_adj_8: 8 independent flows (0->1, 2->3, 4->5, 6->7, 8->9, 10->11, 12->13, 14->15) on disjoint shire pairs. Does aggregate scale linearly?

• disjoint_adj_16: Same but scaled to all 32 shires

• parallel_cols_4: 4 long flows (0->16, 1->17, 2->18, 3->19) down parallel columns. How does that work?

• row_crossing_4: 4 flows (0->7, 1->6, 2->5, 3->4) in the same row that overlap. Does that congest?

Apparently no. No congestion at all. It seems the only congestion I can find is source read bandwidth. As more and more shires try to read from the same source, bandwidth plateaus. But again I cannot tell if this is a read port limitation or L2 only supporting that read bandwidth. And it’s kinda suspicious that the measured max L2SCP bandwidth is around the claimed DRAM bandwidth on the card. Coincidence? Your guess is as good as mine.

If I target Shire 14 as the source (center of the chip). Aggregate bandwidth from L2SCP to all shires should be higher, right? Also no. It actually achieves a lower peak and aggregate bandwidth drops off at the end. NoC packet congestion......?

I am as baffled as you are. New question: How does the DRAM work? Does it follow the same pattern? Let’s just read from the DRAM both in single shire and all shires..

It is surprisingly even. I would expect more performance variance as shires are further and further away from the DRAM. Wherever the DRAM is. But again, the NoC is faster than the DRAM bandwidth (thank god I am not working on a NoC congested chip again). Especially the block diagram in the datasheet tells us nothing about where the DRAM is on the NoC. The best guess I have given the memory read pattern:

Also the Programmer’s reference manual is unclear on how L3 is used. Especially in the context of the TensorLoadL2SCP command.

The TensorLoadL2Scp instruction copies a tensor from memory, bypassing the L1 data cache and the L2 cache, to the L2 scratchpad.

Doing some benchmarking.. we can see that performance drops if the reading set is larger than ~32MB. Which coincides with the default L3 configuration (1MB per shire, 32MB total).

Given we kinda know where the Memshire lives and can observe its effect on bandwidth, and we have a problem getting the full 120GB/s during real workload. Can we do the same trick as some other grid-like AI accelerator companies and make shires send requests to one specific Memshire? So we can maximize memory access pattern and avoid crosstalk...

Not at large intervals, which is expected because that’d be a terrible design

We can find patterns at really small intervals -- at near the cacheline size, we find a pattern of 512B.. so 8 cachelines (cacheline size is 64B).. mapping to the 8 Memshires?

Which if we fold the cacheline sized graph along 8 cachelines. We get the following diagram. That clearly shows different bandwidth to different Memshires, up to 50% difference.

With that knowledge we can hit the specific lines that we know the access goes to specific memshires, loading from a different address to avoid L3 effects. And bingo! We found approximately where each memshire is and the achievable bandwidth from them with only one shire asking for data:

Now the final and obvious question - where does the memshire live on the NoC? ... but before that the other question - is the 8x4 mesh correct? We assume the underlying is 8x4 because that’s what the datasheet and PRM say. We all know these can lie. Let’s do the same simulated annealing but make the topology itself also part of the model. And see if we can predict the bandwidth:

So documents do lie. Turns out it is a mesh... but with missing links (I assume yield reasons, so the topology on your chip might differ), definitely not the 8x4 mesh that you’re lead to believe according to the datasheet product brochure and we can almost perfectly express the bandwidth between shires as a function of hops given the right topology. But that hole at 5,3 feels suspicious. And there’s outliers in the predicted bandwidth. Why does our standard deviation suddenly go from very small to 0.25? Unless... there’s other nodes that exist on the NoC that passes through NoC transparent from/to the shires, but it itself not a compute shire so I cannot measure the bandwidth of it. What if I just add a node there? Does that improve my predictive power?

And now all the outliers are gone. That left me suspecting the 3 empty nodes are actually also routers. Maybe...? But there is not enough topological information to determine their existance or not. By the same approach we can figure out where the memshires lives on the NoC -- the predictive power of the graph is already high, so they must not sit on their own NoC hop in the mest, either outside of the current mesh or they inject themslves into the mesh. Some simulated annealing later... We have the result. Note that I (personally) assumed the injection location is not uniform. Memshire 0, 1 and 7 have 2 locations (because the plots seem to indicate so). But know this is likely wrong and contaminated by L3 access speed as chip floorplan would be insane if designed as so:

And we can update our visualization of Memshire access performance and it looks absolutely beautiful. But this does not sit well with me - you wouldn’t put memory access at the center of the NoC. Maybe it’s L3 working as intended? Instead of shrire -> Victim L3 -> shire (not found) -> memshire it chip dpes shrire -> Victim L3 -> memshire -> shire? Either way bandwidth is bandwidth and that’s what matters for a real kernel.

We can finally figure out the max bandwidth possible against a single memshire. And calculate the actual peak bandwidth achievable to be ~88GB/s on my specific chip.

As a side note. I tried to figure out if I can use latency instead of bandwidth to locate the Memshire. No. Atomics share the same pattern. Presumably the chip uses L3 as the atomic target instead of DRAM. Fair. But apparently atoics have a baseline 120 cycle latency regardless of where they are in memory (plus measurable overhead, but that’s relatively low). And each hop contributes a clean 6 cycle latency increase. Which indicates 3 cycles per direction.

The ET-SoC-1 chip is really nice. It behaves like a flat processor without the NoC being super obvious. You can still see it. And you can still use it. But most applications ought to treat the processor as a flat array of cores and not step onto any footguns. Best of both worlds.

Leave a comment

Purely a community effort. If you haven’t seen them yet, find local events or a Makerspace - it’s amazing how something so small and toy-looking can lead to real-life use cases and a new paradigm of training robots, not models.

Watch Sergey’s talk to find out more!

Subscribe now

Ai Plumbers Fosdem 2026 Lerobot

3.67MB ∙ PDF file

Download

Download

He covers memory-bound bottlenecks, the kernel-builder workflow, reproducible multi-hardware builds with Nix, and practical PyTorch/Transformers integration patterns that reduce setup time from hours to seconds.

Key moments from the talk:
0:00 Intro and speaker background
1:35 Why Hugging Face Kernels matters
2:05 Compute vs memory bottlenecks in deep learning
3:30 Fused kernels and why they speed things up
5:05 Talk agenda and ecosystem overview
5:35 Kernel pain points: fragmentation and long installs
7:12 Supporting older, cheaper hardware for the community
8:18 Goal: from CMake errors to one-line kernel usage
8:54 Kernels + kernel-builder architecture
10:00 Reproducible builds with Nix and support matrix
11:5 Kernel project structure (`build.toml`, sources, ton)
12:23Publishing kernels to the HugHub
13:03 Real-world gain: faster FlashAttup
14:18 Docs, repos, and how tted
16:00 Verifying compatibility and loading kernhon
17:20 Managing local cache with ls`
17:55 Kernelizing PyTorch layers with hpgs
19:23 Transformers integration (`use_ke`)
20:48 Performance chart and closing resources

Hf Kernels Community

3.68MB ∙ PDF file

Download

Download

Subscribe now

They cover agent-assisted kernel development, benchmarking results across model families, and why reproducible kernel infrastructure is key for broader hardware support.

Key moments from the talk:
0:00 Intro and guest welcome
0:28 What Hugging Face Kernels solves
1:53 Can LLMs help write kernels?
2:06 Using Claude Code + skills to generate kernels
3:21 Benchmarking different models for kernel generation
4:30 Agentic workflows and performance engineering
5:16 Supporting older and affordable hardware
6:55 Why infrastructure matters beyond generated code
7:46How Kernels Hub helps developers learn and contribute
9:20 How to get involved with Hugging Face Kernels 10:00 Wrap-up

Subscribe now

​​The AI Plumbers Unconference: San Francisco Edition is an open-source meetup for builders of low-level AI systems to dive into the plumbing of modern AI, from modern data infrastructure to AI accelerators.

Watch #AIPlumbers presentation by Gianluca Guida, Head of Software, Ainekko on Introducing ET-SOC - the fully open source manycore platform, first ever talk introducing it!

Key moments from the talk:

00:00 – 00:25 — Introduction to the topic

00:26 – 01:00 — Gianluca Guida’s background

01:01 – 02:00 — Agenda

02:01 – 03:54 — ET Platform Software

03:55 – 05:32 — ETSOC1 Hardware: The PCIe board — Overview

05:33 – 07:25 — ETSOC1 Hardware: ETSOC-1 — Overview

07:26 – 09:05 — ETSOC1 Hardware: Minion — Overview

09:06 – 13:03 — How can I take part in this project?

13:04 – 15:21 — Q&A

Subscribe now

​​The AI Plumbers Unconference: San Francisco Edition is an open-source meetup for builders of low-level AI systems to dive into the plumbing of modern AI, from modern data infrastructure to AI accelerators.

Watch #AIPlumbers presentation by NVIDIA team on Dynamo, a deep dive into production environment for inference at scale, i.e. both compute and memory demands exploding exponentially.

Disaggregated serving, intelligent scheduling, multi-tier memory management, KV-routing, and high-availability mechanics — all designed to push inference efficiency to the maximum.

This #AIPlumbers talk showcased production-grade engineering: from offline performance configurators that find optimal cluster layouts, to dynamic K8s scheduling that understands physical GPU topology, coordinated multi-GPU serving, etc. Lot’s of clever tricks on handling compute-bound vs memory-bound workloads, I’ve heard people discussing before, but now not in theory! And it’s all #opensource.

And also really hope to hear more at #FOSDEM26 from the Dynamo team - don’t miss it!

Key moments from the talk:

00:00 – 01:02 — Dynamo: Inference at Scale

01:03 – 02:49 — Inference Compute Requirements Scaling Exponentially

02:50 – 05:59 — Dynamo: A Systematic Approach to AI Inference at Scale

06:00 – 08:54 — Memory Management

08:55 – 12:19 — KV Router

12:20 – 15:00 — Production-Grade Serving with Dynamo

15:01 – 16:33 — Offline Perf Configurator

16:34 – 18:39 — Offline Perf Optimizer

18:40 – 26:00 — Topology-Optimized Dynamic K8s Scheduling

26:01 – 29:22 — Fault Tolerance

29:23 – 32:32 — How Dynamo Works

Subscribe now

​​The AI Plumbers Unconference: San Francisco Edition is an open-source meetup for builders of low-level AI systems to dive into the plumbing of modern AI, from modern data infrastructure to AI accelerators.

Watch #AIPlumbers presentation by Tanya Dadasheva, CEO & CO-Founder, Ainekko and Roman Shaposhnik, CTO & Co-Founder, Ainekko - their first presentation to the community and proper introduction of Ainekko. Well, technically it’s the second presentation, the first one was at #RISCV summit!

This #AIPlumbers took place in the industrialized comfort of Studio45 - makerspace in SF, where Ainekko lab is. Nothing shows the vibes of Ainekko more than this - not a fancy office, not an official sales/marketing pitch - here we are with are closest friends - open source community.

Key moments from the talk:

00:00 – 01:19 — Intro

01:20 – 02:04 — Industry take on AI Infrastructure

02:05 – 02:47 — Evolutionary take on AI Infrastructure

02:48 – 03:15 — VC take on AI Infrastructure

03:16 – 04:36 — Evolution of Chip & Framework Codependency

04:37 – 06:43 — Open Community take on Infrastructure

06:44 – 08:52 — From SW to Manycore Architecture

08:53 – 13:10 — RISC-V Manycore LAMP Stack: “Less is More”

13:11 – 13:38 — 100% Evaluation Platform

Subscribe now

​​The AI Plumbers Unconference: San Francisco Edition is an open-source meetup for builders of low-level AI systems to dive into the plumbing of modern AI, from modern data infrastructure to AI accelerators.

Watch #AIPlumbers presentation by John Leimgruber, Community LLM Quantizer also known as ubergarm is one of the most known and productive quantizers in AI world. I’m not joking about productivity - he has about 30TB of quants on Hugging Face, they even started to limmit his uploads (all fixed now, no worries, there will be more!). I wish there would be people with the job “quantizer” on Linkedin so at least we would know how many are there but also I’m not so sure all of them will even have Linkedin. Anyway, I don’t know to many, but I sure know the great ones!

Btw, if you haven’t seen it go watch Iwan Kawrakov talk from last #FOSDEM 2025!

But also do watch John’s talk to learn what does it take and how to start the journey into #quantization, the benchmarking of different quantizations, what metrics to use - what is speed? what is inside the quant? is fp8 a data type or a quantization type? So if you don’t have time to download the whole #ggml github discussions and PRs and grep to figure it all out - listen to somebody who did.

And in the true #AIplumbers tradition it doesn’t stop on SW optimizations - different #HW backends, memory usage optimizations, thermal considerations and more!

Key moments from the talk:

00:00 – 04:30 Personal Background and Journey

04:31 – 7:30 Quant Cooking Quick-Start

07:31 – 10:17 Benchmarking Quantization “Quality”

10:18 – 12:39 Quant comparison for “Quality”

12:40 – 16:15 Benchmarking Quantization “Speed”

16:16 – 18:20 LLM Tensors

18:21 – 20:38 llama-quantize — help

20:39 – 21:57 MXFP-4 Quantization with 4-bit blocks

21:58 – 27:27 GPU Tuning

John’s presentation is here!

​​The AI Plumbers Unconference: San Francisco Edition is an open-source meetup for builders of low-level AI systems to dive into the plumbing of modern AI, from modern data infrastructure to AI accelerators.

Watch #AIPlumbers presentation by Dhruv Diddi, Founder, Solo Tech who walked us trough the demands of #physicalAI from chips to the fine tuning of open robotics models. We all know love #lerobot and SmolVLA models from Hugging Face that bootstrapped that ecosystem and now there is enough substance to innovate, customize and optimise. There are also some demos and interesting numbers in that video - so watch till the end!

And if you have never seen those cute robots check out Dhruv’s #RoboticsGym events or just hang out more at makerspaces!

Key moments from the talk:

00:00 – 00:45 Opening and Purpose of the Talk

00:46 – 1:40 Personal Background and Journey

01:41 – 02:48 Redefining Intelligence and Artificial Intelligence

04:30 – 05:00 From Early AI Models to the Open-Source Revolution

05:01 – 06:20 The Rise of Physical AI and Robotics Challenges

06:21 – 7:26 Usecase: Robotics

7:27 – 10:40 Solo. Open Collaboration and Measurable Impact

10:40 – 12:10 Looking Ahead: The Future of Physical AI

12:11 - 14:00 Q/A session

Subscribe now

Watch #AIPlumbers presentation by Roman Shaposhnik, it raises a lot of questions like what stack can the industry standardize on and making analogy with LAMP stack of the previous era, who is L,A,M and P for the #AIcomputing industry and what role does the #HW play in it.

Questions were raised, community discussions happened and now we coming with some of the answers. And as always we would love community feedback as we don’t want to be developing anything in isolation.

Key moments from the talk:

00:00 – 02:03 From the Internet Boom to the AI Era

02:04 – 05:20 The Rise of Digital Entrepreneurship

05:21 – 08:00 Scaling Up: From Closed Systems to Open Ecosystems

08:01 – 10:30 The Nvidia Monopoly and the New AI Reality

10:31 – 13:10 Rethinking the Modern AI Stack

13:11 – 14:39 What Enterprises Actually Use in Production

14:40 – 18:45 Building the Open-Source AI Infrastructure

18:46 – 21:53 Observations. Moving towards Collaborative and Sustainable AI Systems

Subscribe now

With the next edition coming on 25th of October 2025 (and this time in SF, USA!) let’s hear the advise and insights from the last edition on how to assist the open source community especially in emerging sectors.

It was great to pick VB’s brain on where to find the contributors, what is a role of companies like Hugging Face in seeding the communities and making different use cases more accessible - a great example is the #LeRobot arm and hackathons around it. I’ve seen more and more of these arms in Makerspaces and at different events but what is even cooler - I’ve seen a few startups picking up the SmolVLA model that was developed for this almost toy robot and finetuning it to real life use cases like excavators and warehouse robots.

Of course, we have ported the SmolVLA model to our HW to make the inference locally and we’ll be demoing it at the upcoming AIPlumbers event!

Key moments from the talk:

0:00 – 1:15 | Introduction & Speaker Presentation

1:16 – 3:08 | Work at Hugging Face & the Role of Open Source

3:09 – 5:10 | Engaging with the Community

5:11 – 6:37 | Favorite Use Cases of AI

6:38 – 8:22 | Local Models vs. Saa

8:23 – 11:10 | Robotics & Hugging Face

11:20 – 12:53 | Future & Potential of AI

Leave a comment

How to start in #AI as a developer, not just a user? We are pretty sure a lot of people have asked themselves this question and it's broader than a skills question. We would translate it to "how to find the right community where what you are doing is non trivial and useful but you don't have to do all the lift alone".

Well #opensource is always a good answer. The good news is that there are a lot of really novel areas where you will get the excitement of being on the frontier, being the first. Moritz Thüning's journey is exactly like this:

stumbling upon Tenstorrent community, working with new and exciting HW and porting models in a new and exciting research field of drug discovery - that's a way to do your PhD and beyond! Of course we couldn't miss an opportunity at #AIPlumbers in Berlin to get a personal perspective, so instead of a formal talk, it's a friendly chat and sincere answers.

Key moments from the talk:

00:00 – 01:19 Intro & First Impressions at AI Plumbers

01:20 – 02:33 Moritz’s Background & Journey into AI

02:34 – 04:44 AlphaFold on Tenstorrent Hardware

04:45 – 06:29 Open-Source Culture & Community Experience

06:30 – 08:15 Advice for Beginners in AI + Closing

Subscribe now

Have you ever wondered if it’s possible to do full model development and deployment lifecycle with open tools only? Karsten walked us through different stages with almost real time prerecorded demos - it still takes time, but hopefully one day the process will fit in the length of a talk. And hearing it from a field engineer from Red Hat - you get a feel of it working in production in real world not just on a single laptop.

Key moments from the talk:

1.00 — Introduction. Who is Karsten Gresch?

2.43 — Various stage of model lifecycle that would be demoed and tools that were used

5.37 — Demo 1 - inferencing model locally: Podman AI Lab, Granite open models

8:40 — Demo 2 - app communicating with LLM deployed locally: Quarkus, LangChain4j

15.19 — Demo 3 - Retraining model locally: InstructLab

24.55 — Demo 4 - Running models in production: Backstage

31.12 — Summary

32.15 — Q&A

The presentation slides are available here:

Ai Plumbers Local Llms To Enterprise Ai From Your Laptop To Production

5.41MB ∙ PDF file

Download

Download

Ever wondered how much companies use their own tools? This is an opportunity to look behind the curtains and see all this AI use cases that Hugging Face has implemented internally. Kicking off our event in Berlin, Vaibhav Srivastav (VB) of Hugging Face with most demo intense talk explaining the implementations of those use cases, what models were used (it’s actually quite a variety), what pipelines and some advice on how to start implementing your own use cases. Go try the example from the talk on HF and get inspired!

Key moments from the talk:

0.29 - Introduction of VB and the use of generative AI and ML in the company’s products

2.25 - The 8 useful AI use cases at Hugging Face that we going to deep dive into

4.24 - Demo of Translation use case

6.20 - Summarization of research papers indexed on HF

7.50 - Emoji generation - SDXL LoRA used on the Hub

8.45 - Demo of Semantic Search on Spaces with natural language

11.18 - Demo of Semantic Search on Daily Papers

13.20 - Structured Generation & Parsing - summarize relevant papers from Archive for the use of researches (highest level of automation and one of the coolest use cases from VB perspective)

16.45 - Demo of SQL Generation

20.05 - Presentation of the entire HF Hub acting as MCP server (Model Context Protocol)

22.36 - Recommendations for using generative AI in your own projects

The presentation slides are available here:

Generative Ai At Hf

5.23MB ∙ PDF file

Download

Download

If you know what AlphaFold is (and you should, it won a Noble Prize) and are thinking that this talk requires you to have a PhD in biology, we have news for you! Sure, the creation of the model required a lot of research, but it doesn’t stop there. There are a lot of ways to make your contributions as an engineer improving the way the model is run and the systems it’s running on. And that way you can become part of this cutting edge solution creating protein structures.

Watch Moritz Thüning demonstrate how he ported the model to a new HW architecture. There are definitely hacks required as there is not yet a compiler that can just take any model and run it, but as Moritz demonstrates it’s doable and absolutely worth it! There is a very special energy in taking a fresh new research and be the first one to run it on certain HW. This kind of exercise truly gives you end-to-end experience that #AIPlumbers is all about. And since it all so new there’s a lot of room for optimizations and be the first one to implement them!

Key moments from the talk:

0:28 — Introduction to the TT Boltz project (running AlphaFold on Tenstorrent) and how Moritz ran into Tenstorrent and decided to do this project

3:22 — Dataflow architecture - is it a successful paradigm?

5:15 — Tenstorrent hardware overview

9:27 — Biology side of things - protein folding problem

10:23 — Results and architecture of AlphaFold 3 (very weird diffusion model)

11:17 — Why Boltz exists - open source licensing restrictions of AlphaFold

12:08 — Profiling runtime on CPU to find modules with highest memory and time complexity

12:46 — Rewriting pytorch modules (performer and diffusion) in tt-nn + rapper, integrate it back for existence proof until there is a proper complier

14:22 — Results - Prediction of a protein

14:51 — Performance across different hardware

15:58 — Triangle Self-attention (in Performer) and possible optimizations to fit it in the chip, trading memory complexity for time complexity, data/model/tensor parallelism

18:18 — Join the work on github!

The presentation slides are available here:

Tt Boltz Berlin

4.32MB ∙ PDF file

Download

Download

Community choices, perfectionism overhaul and the locally run demos - Xuan-Son Nguyen demonstrating how vision support was added to llama.cpp for multimodel use cases, the obstacles and the clever hacks. Still work to do - go try it on Hugging Face Spaces (of course) or locally and contribute to llama.cpp!

Key moments from the talk:

0:55 — Demo running locally llama-server with Qwen 3bn omni model with image and audio input

3:21 — Introduction: who is Xuan-Son Nguyen

4:10 — A little bit about history - how multimodel works

6:10 — History - adding and removing multimodel (LLaVA) support in llama.cpp

9:21 — History - what caused the problems in the llava.cpp / clip.cpp implementation

10:45 — How to fix it?

12:08 — Enter libmtnd

13:12 — libmtnd architecture

16:50 — libmtnd: minimal, simple, well-documented API (adding audio support didn’t require API change!)

17:30 — LM Studio is one of the earliest adopter of libmtnd

18:10 — Demo mtdt-CLI

19:14 — Bring this work to llama-server (some functionality)

21:55 — llama-server WebUI

23:58 — Viral demo - try it!

25:10 — TODO

The presentation slides are available here:

Ai Plumbers 2nd Edition

1.77MB ∙ PDF file

Download

Download

In this conversation, Tanya sits down with Shubham Saboo, Head of Developer Relations at Tenstorrent, to dive into what it takes to connect developers, open-source contributors, and AI innovators across the stack - from conference insights to real-world engagement tactics.

00:30 — What does “community” mean for the Head of Developer Relations at Tenstorrent, and how did Shubham get started in this role?

02:08 — What conferences does Shubham attend to learn about model enhancement and the rest of the stack?

03:35 — What has Shubham learned from different conferences related to open source?

04:54 — Meeting developers, vendors, and model builders where they are

06:33 — A detailed look at Tenstorrent’s community-building approach

07:40 — Hacks and tips to engage people

10:08 — What excites Shubham, as Head of Developer Relations at Tenstorrent, about working in the AI ecosystem?

Whether you're building AI tools, contributing to open source, or simply passionate about community in tech — tune in and share with those shaping the future of AI, one connection at a time.

1:00 — Felix’ background and big start in the world in OpenBLAS and Formula1

1:53 — Engineers having a real job in AI, finally!

2:50 — Open source plays a major role in fostering discussions like this and for people arguing about compilers

3:17 — “When you build something, you want it to be the best” - that’s the honesty OSS gives you

4:20 — A place to think about end-to-end problems - putting all the parts of the puzzle together

5:06 — The most mind-blowing CPU architecture for Felix

6:30 — Applying AI to chip development

7:33 — What are compiler friends talking about

8:32 — “Machines building machines” = AI driven silicon design

9:03 — “Eating your own dog food” and doing it in open source

Felix’s insights are both experienced and bold — a reminder that if you’re building, build to be the best. Share with fellow nerds who love thinking at this level.

Key moments from the talk:

2:07 - About the AI Alliance...

3:27 - Do I Need a Ridiculously Large Closed Source Model?

4:24 - An (Embarrassingly Quick) Overview of LLM Function Calling

10:21 - Overview of Chain of Thought

14:29 - Implementation - The Code

15:26 - Implementation - The Demo

15:45 - Do Math

16:30 - Hot Take - Your Laptop Is Not An LLM Server

18:25 - Hosting your LLM

The presentation slides are available here:

Trevor Grant @ AI Plumber Ghent

2.42MB ∙ PDF file

Download

Download

Leave a comment