Hello everyone！
I am currently exploring the performance of large models in understanding knowledge in specific domains, and attempting to construct a knowledge framework similar to what humans establish when learning a subject. This understanding does not need to be flawless, but it should provide a comprehensive grasp of the core concepts and structure of the subject.
Research Questions:
Has the academic community conducted research on the extraction of knowledge understanding from large models for specific content?
Is there any professional terminology to describe the output process of this knowledge framework?
Purpose:
My goal is to extract the understanding of a subject or theme by a large model and hope that this understanding can be represented in the form of vectors or text, which can then be "fed" to other AI systems to achieve knowledge transfer and application.
Request:
If you are aware of any relevant research, could you share some professional terminology or key concepts? If possible, please attach some relevant academic papers or resource links; this would be of great help to my research.
Application Scenarios:
The application scenarios I envision include, but are not limited to, using the output of large models as input for other AI systemssing the output of large models as input for other AI systems to expand the knowledge scope and application capabilities of AI systems.
Thanks:
I express my heartfelt gratitude for any suggestions, guidance, or resource sharing, as this is crucial for advancing my research.
Looking forward to everyone's replies and discussions!
Your research into how large language models (LLMs) understand and represent domain-specific knowledge is an exciting area. Here's an overview of relevant concepts and research directions:
Key terminology and concepts:
1. Knowledge Distillation
2. Prompt Engineering
3. Few-shot Learning
4. Zero-shot Learning
5. Concept Bottleneck Models
6. Probing Tasks
7. Semantic Embeddings
8. Knowledge Graphs
9. Ontology Learning
Relevant research areas:
1. Interpretability of LLMs
2. Transfer Learning in NLP
3. Structured Knowledge Extraction
4. Neuro-symbolic AI
Recommended papers:
1. "Language Models are Few-Shot Learners" (Brown et al., 2020)
2. "Extracting Hierarchical Knowledge from Language Models" (Zhang et al., 2022)
3. "Probing Large Language Models for Conceptual Understanding" (Li et al., 2023)
For application scenarios, look into:
1. Domain-specific chatbots
2. Automated knowledge base construction
3. Curriculum learning for AI systems
To further your research, consider exploring:
1. ACL, EMNLP, and NeurIPS conference proceedings
2. ArXiv's Computer Science > Computation and Language section
3. Journals like Transactions on Knowledge and Data Engineering
These resources should provide a solid foundation for your investigation into LLMs' knowledge understanding and representation.
Dear Yh Lu ,
Knowledge distillation transfers knowledge from a large model to a smaller model without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device)... The idea of using the output of one neural network to train another neural network was studied as the teacher-student network configuration. Hope below quoted sections are relevant and help your research.
https://ai.stackexchange.com/questions/46452/exploring-large-models-knowledge-understanding-output-and-academic-research-on
Regards,
Shafagat
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs