Reinforced Knowledge

When the translator starts solving the problem

Mon, 20 Jul 2026 17:10:00 +0200

The project stopped being a throughput project when I opened one of the fast outputs and found an answer.

I was translating a large reasoning dataset into several European languages. The serving stack looked healthy. Requests completed. The inference server returned finish_reason = stop. Output files had the expected number of rows. The translated text had plausible length. Everything looked fine from the outside. Then I read the samples.

Inside some of the rows, the model had silently changed jobs. A source example asked for a proof, a program, or a mathematical derivation. The translation prompt asked the model to translate that example. The model sometimes followed the instruction embedded in the source and solved the problem instead. In other cases it restated the prompt, wrote a concise answer, or replaced a long reasoning trace with a much shorter piece of helpful prose.

Anatomy of verl, the RL post-training framework I lived in

Mon, 01 Jun 2026 21:45:17 +0000

I came to verl for a research project: RL on function calling. I needed custom environments where the model interacts with tools and is rewarded on the call it makes. The capability to have custom rewards and eventually the room to modify or write the algorithms. Also, since I was familiar with them, I wanted FSDP and vLLM if I could get them. Oh, and not to forget, the capability to do long horizon training.

Install flash-attn without crying while using uv

Tue, 23 Sep 2025 01:15:24 +0000

Introduction

uv’s documentation about build isolation is already great and has everything you need and I recommend and suggest to read before anything else. Read the whole documentation even, especially if you’re just starting with uv.

I’m writing this article since these last days I had to either install flash-attn as a wheel directly or build it from source and since I struggled different times, I thought of noting down my path so that it stays fresh in my mind. Unfortunately, at the time I didn’t read the documentation when going through all of this cause I thought that I could figure it out quickly. Well…

A story of using langchain/langgraph

Sat, 19 Jul 2025 02:08:30 +0000

Hi everyone!

This is going to be a short post, contrary to what I do usually, but I was going to reply to this Reddit post Disadvantages of Langchain/Langgraph in 2025 and found that my comment was too long and decided to make a Reddit post in and of itself so that maybe more people can see it and maybe it’ll resonate with others, and maybe we can gather more stories about using langchain/langgraph or other libraries or just the experiences of developers working in this space.

Python Project Management and Packaging: PEP 751 update and some of the remaining issues of packaging

Fri, 02 May 2025 06:50:50 +0000

My first two articles (part 1 and part 2) on Python project management and packaging gathered a lot of interest and I thought they were comprehensive enough for me not to come back and update them for a moment.

But, about a month ago, PEP 751 – A file format to record Python dependencies for installation reproducibility, was accepted, 31 March 2025. At the time of publishing my first two articles, I didn’t think that lockfiles would get standardised and I didn’t know about the ongoing effort.

A Comprehensive Guide to Python Project Management and Packaging: Concepts Illustrated with uv - Part II

Wed, 20 Nov 2024 19:46:43 +0000

In the first part we delved into the nitty gritty details of initializing and managing the dependencies of a Python package or project. This covered what is a Python package as opposed to a plain project (and other nomenclature), what is defined in the standard (e.g., project, build-system, tools and dependency-groups tables in pyproject.toml through the PEPs 518, 621 and 735) and what are some common features that tools provide on top of the standard (e.g., lock files, editable dependencies).

A Comprehensive Guide to Python Project Management and Packaging: Concepts Illustrated with uv - Part I

Wed, 06 Nov 2024 00:04:38 +0000

The goal of this guide is to provide a comprehensive guide to Python project management and packaging.

We’ll explore concepts in the standard, like the different tables in pyproject.toml by revisiting the PEPs that led to what we have today. We’ll explain what was used before, why it needed to change, and how the changes provided by the PEPs solved the issues. This walkthrough of the historical context is important to understand current practices.

Deploying a Streamlit app on AWS EC2 (with your own domain name)

Wed, 30 Oct 2024 23:08:28 +0000

The goal of this article is to provide a clear step-by-step guide starting from the fundamentals for deploying a Streamlit app on an AWS EC2 Instance and use your own domain name as well.

So this guide will not only help you if you’re a beginner in AWS but also in deploying Streamlit apps.

In this article I’ll deploy my inflacon project on my domaine inflacon.com.

Though in this article we’ll go from having nothing, not even an AWS account to accessing your app using a domain name, this article is not intended to be a course on the basics of AWS that we’ll be using, on Streamlit, or any other stuff we’ll be using.

Position Information in Transformer-Based Models: Exploring the main Methods and Approaches

Tue, 25 Jun 2024 22:09:25 +0000

This post will only focus on some position encoding, namely, sinusoidal, T5 bias, ALiBi and Rotary. It’ll also discuss no position encoding (NoPE) and FIRE (functional interpolation of relative position encoding) This choice was made because the first methods are the most present in comparative benchmarks while for NoPE and FIRE, I found the ideas and the maths behind them interesting so I wanted to talk about them as well. These methods cover a good range of the ideas used in position encodings so understanding them gives insight into many other position encoding methods that were developed at the same time.

Sparse Transformers

Mon, 03 Jun 2024 17:41:18 +0000

Paper: Generating Long Sequences with Sparse Transformers

Before diving into this paper, I’ll have to say that this article might change in the future to include from scratch (but still relying on sparse backend operations) implementations of the theoretical elements introduced in this paper.

In this article we’re not going to delve into the following elements introduced in the paper:

The recomputation of attention matrices to save memory
Fast attention kernels for the sparse attention mechanisms

We’re only going to tackle, from a theoretical perspective only, the sparse attention mechanisms. Reading this paper I found many painpoints and I think I can clarify some of them for the readers that are interested in this article.

Decoder-only Language Models Architecture Evolution (Part I)

Thu, 30 May 2024 17:53:57 +0000

(Part I)

Modeling language can be done using a wide variety of techniques (e.g. Markov chains) and in this article, we’re going to focus neural networks approaches and specifically the decoder-only architectures, which are based on the transformer architecture introduced in the paper Attention Is All You Need, in 2017. If you’re interested in it, you can read my deep dive into the architecture and the design choices here.

The decoder-only language models started with OpenAI’s paper Improving Language Understanding by Generative Pre-Training

Transformers: Attention Is All You Need

Sun, 24 Dec 2023 22:01:43 +0000

All the code in this article can be found on ReinforcedKnowledge/deep-learning-from-scratch/transformer

Background

The Transformer comes as an answer to the sequential computation constraint that recurrent neural networks (RNNs), long short-term memory neural networks (LSTMs) and gated recurrent neural networks (GRUs) suffer form. The Transformer leverages the attention mechanism that allows to model dependencies between positions in the input and/or output sequences regardless of their distances.

Model Architecture

Attention

The description of attention by the authors of the paper is clear and concise, hence we’re citing it here:

About

Tue, 17 Oct 2023 15:24:56 +0000