It was December 2017, the penultimate semester of my undergrad was over and I was placed in a great company hrough the campus placement. However the feeling of not having contributed much to the society was nagging me somewhere. While I was browsing through my IITD mailbox, I jumped to take part in the first opportunity I found. Several projects were being taken up by The National Trust of India and IITD in a joint venture. I signed up for it.
It turned out among the many projects to spread awareness about autism and promote inclusiveness, one of them was to develop an Android app. This app would help children children having autism communicate better using a flashcard like interface. I, along with a team of 5 others were selected for this project. It turned out there were already some apps existing for this very same problem, but they were not “Indianized” enough. The requirement was to have an app that would have the local variations of the common words (eg. “biscuit” instead of “cookeies”) and have its UI primarily in Hindi. We also wanted the app to be free of charge so that families from any financial backgroud could use it.
Planning the app
Our first step was to understand the problem and the requirements of the app. Naturally we decided to meet the teachers of a special school, Action For Autism. We went there for a day, observed the kids and how they interacted with the teachers. We then also used the current similar apps in the market and took feedback from the teachers and parents. As a result we had several points, features and a good idea of how we should proceed with the development.
We realized we needed to solve the following:-
- Have an app which helps kids communicate by tapping on word tiles in app
- Keep the words in India context
- Support the Hindi language for UI
- Be customizable
We brainstormed that these could be solved by:-
- Having a decluttered and very simple UI
- Have a default Text to Speech (TTS) for each tile to speak the word out.
- Do a comprehensive study of commonly spoken words and actions by Indian kids and include them in the app.
- Allow kid’s caretaker to add new word tiles
- Allow chaning of picture on the tile
- Allow recording of voice for a tile
I being the only one with a Computer Science background in the team decided to take up the responsibility for overseeing the app development.
The app can be broken down into the following main components:-
- The user interface
- Text to Speech
- Data storage
- UI customizations
- Content customizations
The user interface
The other members in my team were primarily responsible for brainstorming and coming up with a simple, intuitive UI. We came up with some sketches and Parth Dixit also coded up a sample screen in Android Studio
The word tiles and the folder tiles are separated. The word tiles are places in the upper half of the screen. Pressing them plays the word. Pressing a folder tile, on the lower half of the screen, opens the folder and displays its word tiles on the upper half. This is just like a normal directory structure.
Settings menu, with all the options. (The child mode option is missing here)
The dialog box allowing user to create a new tile
This was the UI idea we started with and built over.
Text to speech
The next important thing to get right was the Text-To-Speech (TTS). We wanted this to have an Indian accent so that Indian kids could comprehend it easily. It turned out the out of the box TTS system by Android satisfied our needs for this.
This was enough to speak out english words with an Indian accent.
Apart from the default TTS, we also wanted to allow the user to record custom voice for the tiles. But now consider this: The user creates a sentence by pressing some tiles with default TTS and some with a custom recording. We need to speak this sentence out too. For this I wrote a
SpeakSentence wrapper. This would be passed a stack of the word tiles and would alternate between custom and default TTS as needed.
The data is stored locally using the SQLite DB provided by Android. I created 2 classes,
DBHelper for the same.
DataProvider initializes the database when the app is first launched. It reads the words from
.txt files. A file for each folder. Each file having comma separated words. The other parts of code interact with
DataProvider and it acts as a wrapper over
DBHelper interacts with the SQLite DB and runs the SQL commands as required. Some functions are:-
The settings page of the app allows user to make some basic customizations. We allow 2 types of customizations:
- Font size: The user can select among 3 font size settings, making the text small or bigger as per convenience.
- Speech rate: This is the speed at which TTS speaks out the words. This again comes with 3 values among which the user can select.
The value selected for these customizations is stored in
SharedPreferences. The interaction with
SharedPreferences is done by the
Utility class. While rendering the screen or speaking out TTS, the
Utility class is again consulted for these values.
An important feature of the app would be to allow the users to customize the default words and folders present in the app, and also add more of these. On the user interface aspect, we decided that when the child mode is disabled, long pressing any tile would present a pop-up menu. This would allow the user to change the tile text, image, sound or delete it completely.
Tile edit box
Edit user flow
To allow these customizations, we needed to modify the data stored in the SQLite database as discussed before.
It was important that the app have text in hindi. We initially ad just the words and folders in hindi, but upon taking feedback from teachers, we were told they really need an app that was completely in Hindi. This was a mojor requirement and no app in the Play Store that satisfied it. Hence we were determined to come up with a solution for this too.
Going by the recommended route, I created a new
values folder with the name
values-hi with a
strings.xml file in it for hindi translations. Then using the
Translations Editor in the Android Studio, I was able to add the translations for all the required sentence in the app UI.
The option to select a language was added in the settings menu and this too was stored in
SharedPreferences in with the
Utility class being a wrapper over
Whenever the app is launched we set the locale using
The strings are then accessed by
This was a great project. We created an app which will be used by many kids who need such a solution. We learnt the importance of feedback while developing a product and having a very clean and simple UI.
I’m really thankful to my team, PVM Rao Sir and The National Trust for this opportunity.