Natural Language Processing (NLP)


Natural language processing (NLP) is the use of statistical techniques, modeling, and machine learning tools by computers to process text-based datasets and understand human language.

  • is a branch of artificial intelligence that analyzes, understands, and generates the languages that humans use naturally
  • foundation is from statistical patterns of data based on massive amounts of text
  • many tools leverage NLP to create the information that can be used when we interface with LLMs
    • data in these LLMs uses patterns learned from data and information the model has been trained on
  • enables Named Entity Recognition (NER)
    • lets us label and find people, places, and more
    • NER tools/libraries allow to train models with custom entity training
      • allows people to train or teach the model to look for their propriety words or items