Interactive inference entry point
An entry point. Running python talk.py calls talk.main at line 30.
0 of 1 functions, classes and methods here are reachable from train.main, and 1 from talk.main.
config.py, checkpoint.py, generate.pysys, numpy| Concept | What this file does | Lines | Status |
|---|---|---|---|
| Random choice | Calls it | 9 | Implemented |
Loads a checkpoint and runs the prompt loop; entry point at lines 29-30
talk.maintalk.<module>checkpoint.load_checkpoint, generate.generate, tokenizer.Tokenizer.decode, tokenizer.Tokenizer.encodeimport sysimport numpy as npimport configfrom checkpoint import load_checkpointfrom generate import generatedef main(prefix="checkpoints/model"): params, cfg, tok = load_checkpoint(prefix) rng = np.random.default_rng(config.seed) print(f"loaded {prefix} (V={cfg.vocab_size})") print("Type the start of a text and press Enter. Empty line quits.") while True: try: prompt = input("> ") except EOFError: break if prompt.strip() == "": break ids = tok.encode(prompt) out = generate(params, cfg, ids, config.max_new_tokens, config.seq_len, config.temperature, config.top_k, config.top_p, rng) print(tok.decode(out[len(ids):])) # only the new tokens, the prompt is already on screenif __name__ == "__main__": main(*sys.argv[1:])