I am starting some research on a project that involves navigating the web with speech feedback from SAPI. This application will be used by the blind and visually impaired to navigate the web with full speech feedback of current selected object and the ability to read the entire page and fill out forms. I have a mass amount of experience in SAPI and the accessibility word, however I’m not so familiar with HTML and the best approach on taking on this project and i would like to ask for guidance from someone that may have experience in this area. The basic concepts, which I’ve already explained a little is to enable a blind user to use this standalone product without a screen reader and have the ability to navigate the web page with a keyboard and receive speech feedback from a synthesizer i have created. I can use the Accessibility API’s built into windows to display location for low vision users.
Thank you!
I have come to the conclusion in order to get a talking web browser i actually had to hook an instance of the WebBrowser Control with Set setwindowshookex. In my setwindowshookex callback i monitor for the event: OB_FOCUS (focus changed, could be keyboard or mouse) i then pass the object instance that has focus to the AccessibleObjectFromEvent API. Next, i get the IAccessible interface to this object and then enumerate any children if present and compare them against a known object list that relates to HTML/Web objects to use the proper class that will parse this field and announce text if available, if the user is in an editable field, focused buttons, etc… I also use the IAccessible class to get the objects location and present location to the user by drawing a rectangle on the current focused object. I have also created a class that extracts just link, text, removes images and more to make for an easier, more enjoyable web experience for people with visual impairments. I still have some quarks to work out, but progress is moving forward!