RESEARCH

Acoustic & Speech

Investigates various Speech signal processing schemes for acoustic modeling so that more robust speech recognition can be achieved. Our aim is to perform the state-of-art research providing effective means for achieving:

Context Awareness System based on Acoustic & Visual

작성자 관리자 날짜 2021-04-08 21:49:18 조회수 3

Context Awareness System based on Acoustic & Visual

 

     Contents

1. Preprocessing

2. Main algorithm and principle

3. Application or Demo

 

1. Introduction       

  This is the context awareness system based on acoustic and visual data in mobile platform. By analyzing the acoustic and visual source from the microphone and camera of the smart-phone (mobile device), we can get some cues for context. The Environments we can recognize are followings:

 

    - 10 Acoustic environments

 Environment

Description

Babble

many people are saying in crowded places(cafeteria or coffee shop)

Car

bus, automobile, street(roadside)

Moving

moving in closed space (pocket, bag)

Music

pop, classic, rock

Noisy

high level audio

Office

printing, typing keyboard, phone ringtone

One speaker talking

only one person is saying(presentation, lecture)

Silent

no talking, silence

Subway

platform and inside of subway

Water

washstand, sink, shower room

 

    - 4 visual environments

Low intensity

Face

Indoor

Outdoor

Moving Detection

 

                                     

2. Main algorithm and principle        [Go Top]

    - Acoustic

        1. Noisy environments detection using the average value of log sound energy

        2. MFCC feature extraction for context awareness

        3. GMM(Gaussian Mixture Model) for the classification

    - Visual

        1. Low intensity : using the image intensity, light detector

        2. Face: skin region detector

        3. Indoor/Outdoor : Nearest neighbor classifier using color/texture features

        4. Moving detection : Neural network classifier using the optical flow for feature

- high speed in mobile platform for real-time

   1. Dynamic mixture GMM for acoustic modeling

   2. Recognition process scheduling

    - System Architecture

 

 

3. Application or Demo       

    

댓글 (0)

등록된 댓글이 없습니다.
작성 권한이 없습니다.