Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Initialize with a pre-trained model.
  2. Prepare a labeled dataset for the question-answering task
  3. Train the model using this dataset, adjusting all layers.
  4. Try different learning rates to avoid catastrophic forgetting and also to avoid over-fitting.

Types of Fine-tuning:

  1. Standard Fine-tuning as I mentioned its steps previously.
  2. Feature-Based Fine-Tuning: This involves using a pre-trained model as a feature extractor and then using a smaller model as a classifier. The advantage of this approach isĀ  it's computationally efficient and less prone to overfitting but also it might be weak in classification.
  3. There are other types of fine-tuning but they may not be suitable for your project

Learning ProgressĀ 

  1. Natural Language Processing (NLP) course.
  2. Studying more about the Langchain framework
  3. Study React.js and Nest.js
  4. Learn more about Blockchain and
  5. Learn more about RAG and fine-tuning