Twitter bot for Art of War -- Part 1

This is the first blog in the series see the other parts here:

• Part 1

• Part 2

• Part 3

Data Collection

• This was the easy part, downloaded the book text from https://classics.mit.edu/Tzu/artwar.html

Data preparation

• Steps:

  • Removed copyright disclaimers

  • Removed empty lines

  • Merged paragraphs into one line

• Cohere expects a text file as training data with a separator in between, used vim to add that file and do the training.

• I also tried the same things on ai21.com, they expect prompt-completion pairs in format called “jsonl”. It’s a text file where each line is a json string.

• I made each paragraph the prompt for next paragraph. Here’s the code to generate the traning data, and here’s the code to validate a jsonl file.

Training

• Training was easy, you needed to upload the training data, select some parameters (tbh I don’t entirely understand these parameters), and queue the model for traning.

• I checked after an hour or so and my model was ready

Output

• The output was meh! This the completition which Cohere gave me

  

• ai21 wasn’t great either, it wasn’t producing anything :(

Turning this into a bot!

• Couldn’t reach this step

Next steps:

• The training data can be improved, instead of full paragraphs as prompts and completion, going sentence by sentence might lead to a better result.

• Also want to try the same things for GPT-3 models.

• Want to try out https://github.com/minimaxir/aitextgen model as well