Non-Fiction LLM Fine-Tuning

I have a sizeable collection of text data drawn from my personal authorship of technical, scientific, philosophical, and legal works, and I want a large language model fine-tuned on this corpus so it can generate accurate, well-structured non-fiction content on demand in my authorial voice. It is a central requirement that the content generated by the fine-tuning will not be and cannot be reasonably confused with general AI LLM generated output text. The base model can be an open-source option such as Llama, Qwen, Llama-2 to be used on Fireworks.ai.

Required Qualifications
• Proven experience fine tuning open weight LLMs (LoRA/QLoRA, PEFT, TRL, Axolotl, Unsloth, etc.)
• Strong dataset curation + instruction tuning background
• Experience evaluating style sensitive generation (held out testing, overfitting controls, memorization checks)
• Experience with privacy sensitive or client-controlled workflows
• Preferred: experience with one of legal, technical, scholarly, or literary corpora.

You’ll 1. start by using a corpus I have created of 150 pages, approximately 100,000 words, (it is not chunked or randomized, and if needed, deduplication, train/validation split), then 2. set up the training pipeline with Hugging Face Transformers, PyTorch, and, if helpful, parameter-efficient methods such as LoRA or QLoRA. 3. Once training is complete, I’ll need the model evaluated for factual consistency and style alignment—automatic metrics (perplexity or BLEU) are useful, but a few human-readability samples will help us judge real-world quality.

This project requires only one LoRA/QLoRA adapter capturing my author’s unified nonfiction voice. The contractor does not need to produce genre specific adapters or maintain strict genre separation. A small fiction corpus may be included within the 150 pages and excluded based on your recommendation, but it does not require a dedicated adapter.

Delivery Time: 30 Days or less, and lesser days will be given greater weight in the decision making as to the best service provider.

Deliverables
• Fine-tuned model weights and tokenizer
• Reproducible training scripts/notebooks with clear comments
• A short README explaining environment setup, how to continue training, and how to run inference
• Sample generation outputs demonstrating the model’s strength across my four focus areas
•Deliver all code, configs, datasets, adapters, and documentation
•Ensure strong confidentiality, reproducibility, and IP protection, where my corpus cannot be reused except for my project.

I’ll provide the raw text and any additional domain guidelines you need. Please outline your proposed workflow, hardware expectations, and an estimated timeline so we can kick things off quickly.

Optional Payment Structure Contractors may choose:
• 20% of project price in cash; and,
• Up to 80% as a capped credit toward U.S. patent related legal services rendered by client who is a licensed patent attorney (subject to conflict clearance and a separate engagement agreement). Third party fees (USPTO fees; drawings; searches; cloud/GPU costs; platform fees; etc.) are excluded from service credit.

Preferred Skills:
• LoRA/QLoRA fine tuning
• HuggingFace Transformers
• PEFT / TRL / Axolotl / Unsloth
• Python + PyTorch
• Dataset engineering
• Instruction tuning
• Evaluation of style fidelity
• Overfitting and memorization control
• Model reproducibility