Xanathar's guide has rules for this in the Time to stop section under Training. It states the following:
With enough free time and the services of an instructor, a character can learn a language or gain mastery of a tool.
Resources. Receiving training in a language or tool usually takes at least 10 weeks of work, but this time is reduced by a number of weeks of work equal to the character's Intelligence modifier (an Intelligence penalty does not increase the necessary time). Training costs 25 GP per week of work.
You can easily edit it and listen to conversations that count for a few days of work for this "time out" activity. This is, however, do not legal in AL (since that would be a rule of the house, which are not allowed). Otherwise, your DM must always explicitly approve this change (obviously).
In the same way, your deputy minister might decide that (whether listening to conversations reduces or not the required downtime), the closer you approach the end of your language learning process, the more you understand conversation elements. Again, this is not supported by RAW, so it's not legal in AL.
Also, keep in mind that, according to XGtE, you should avoid excessive breaks between periods of downtime:
The days of an activity do not need to be consecutive; you can spread them over a longer period than that required for the activity. But this period should not exceed twice the time required; otherwise, you should introduce additional complications (see below) and possibly double the costs of the activity to represent the inefficiency of the character's progress.