Natural Language Processing (NLP)
Natural language processing (NLP) is the use of statistical techniques, modeling, and machine learning tools by computers to process text-based datasets and understand human language.
- is a branch of artificial intelligence that analyzes, understands, and generates the languages that humans use naturally
- foundation is from statistical patterns of data based on massive amounts of text
- many tools leverage NLP to create the information that can be used when we interface with LLMs
- data in these LLMs uses patterns learned from data and information the model has been trained on
- enables Named Entity Recognition (NER)
- lets us label and find people, places, and more
- NER tools/libraries allow to train models with custom entity training
- allows people to train or teach the model to look for their propriety words or items