Work / Model research / The Noir Recording Chamber

R&Bot

A lyrics-oriented transformer study. Text generation, not audio generation.

Owner-confirmedSource private

Contents

What’s in this book?

Three passes: a minute, how it works, then the deep dive. Stop after any one.

  1. Pass 1 Overview 1 minute
    1. What is R&Bot?
    2. What can this page show, and what can’t it?
  2. Pass 2 How it works 2 minutes
    1. What would its output look like?
  3. Pass 3 Deep dive 2 minutes
    1. What has to exist before a model card?
    2. What would Ram test next?

Pass 1 of 3

Overview

What R&Bot is, and how much of it can honestly be shown today. 1 minute

Pass 1 · Overview

What is R&Bot?

R&Bot is a 33M-parameter autoregressive transformer, implemented and trained from scratch.

R&Bot was trained for four epochs on an approximately 50M-token mixed-domain corpus, staged from narrative text and poetry to genre-focused lyrics, with section tokens ([VERSE], [CHORUS], [BRIDGE]). Owner confirmed

My contribution

What I built
A small transformer, written and trained from scratch, that learns the shape of a song as well as its words: section tokens mark where a verse, a chorus and a bridge begin, so the model is asked to produce the parts of a song in order rather than one long run of lines.
The tradeoff I chose
A model of 33 million parameters has little room to discover a song’s structure by itself, so the structure is written into the training text instead: each section carries its own token, and the model learns to produce the sections in order.
Where it stands
The figures above come from my résumé. The source, tokenizer, training logs and checkpoint are not public yet, so this page shows a hand-written sample of the format instead of model output, and it makes no claim about how good the lyrics are.

Pass 1 · Overview

What can this page show, and what can’t it?

These are my own figures. The source, tokenizer, logs and checkpoint are not public, so nothing here is reproduced, and no output-quality claim is made.

Pass 2 of 3

How it works

What its output is meant to look like. 2 minutes

Pass 2 · How it works

What would its output look like?

These lines were written by hand to show the structure. They are not output from the model.

Illustrative sample, not output from an audited checkpoint

[VERSE]

Streetlight hums a minor key

Half the city still awake

You left your jacket on the chair

I left the porch light on, for your sake

[CHORUS]

Slow down, slow down, the night is long

Say it low so I can hear

Every hour is an open door

Stay until the sky is clear

[BRIDGE]

If the morning asks for you

I will tell it you were here

These lines were written by hand for this site. They show how structured lyric text would read. They are not model output, and no setting on this page changes them.

Pass 3 of 3

Deep dive

What is still missing before the model itself can be shown, and what comes next. 2 minutes

Pass 3 · Deep dive

What has to exist before a model card?

What has to exist before a model card, or a live demo, can be published:

Model card requirements and their current state
RequirementState
Model source and config Not yet located
Tokenizer and vocabulary Not yet located
Parameter count (33M, owner-confirmed), by a stated method Method not published
Dataset provenance and rights Not reviewed
Training logs and evaluation Not yet located
Checkpoint and hash Not yet located

Pass 3 · Deep dive

What would Ram test next?

Locate the checkpoint, write the model card from what it actually supports, then decide whether a small isolated inference worker is justified. Until then this room stays a labeled sample.