1. Motivation
- Needs for contextual information
- Recent advances in NLP have led to the proposal of multi-modal models such as VisionLLM and OpenFlamingo
- Currently there is a lack of LLMs specifically designed to deeply understand user context

2. Research goal and issue
- Goal : Develop a mechanism to provide contextual information
- Issue
- Human Activity Recognition / Location-Based System : the classification model used is relatively old and has lower activity recognition performance. LBS systems have low accuracy, not applicable in situations indoor/outdoor mixed environments
- Information fusion : CAAFE-information fusion method is laborious or inaccurate
3. Approach
- Develop a pipeline for injecting contextual information
- Data collection and develop baseline model
- Fusion techniques of optimal descriptive formants for improving LLM response
- Scenarios
- Restaurant Recommendation
- Safety Alerts
- User Aid based on weather


4. Result on scenario1
- Retrieve restaurant information near the user's current GPS coordinates using a map API, and filter this information based on the user's specific query
- Scroll 15 restaurant based on current location
- Distance is derived based on User's GPS coordinates between restuarant location
- Use hte Maps API to search for nearby restaurant information and convert their data into contextual data to enhance the responsiveness of your LLM model
- Context information is crucial for improving the accuracy and relevance of language model responses
- The utilization of context information is necessary for answering difficult questions, such as understanding the specific food a user wnats to better meet their needs
- To prevent hallucinations from unclear details, responses should more effectively utilize exact restaurant information such as distance, address, and restaurant category.

