I have a Garmin eTrex Venture HC for sound recording.
I use it to "track" where I walk, so I can review my actual route later on. I input waypoints where I take recordings. I often announce the waypoint into the mic for easy matching up later, and put those GPS coords into the sound file inventory I keep.
For each day, I transfer the track and waypoints data back to my computer via USB, view them in the "MapSource" software bundled with the unit, and save the data to the same folder as the sound .wav files.
I can go back and view track/waypoints for any day, and know where I recorded each .wav, etc. It removes discrepencies when relying on memory alone to reconstruct where I walked, and where the bird territories actually were, etc.
The default map is bare -- only major roads, no terrain. Not a problem for me since that's not how I use the unit, but keep that in mind when viewing ads (the Venture HC ads show it with $100 maps, not the free default map). If I want a detailed physical terrain view (I always do), I use the MapSource software option to view the track data in Google Maps.
The features I use may be available on less expensive units, but I don't know for sure.
I use rechargeable double-A batteries; the battery life is plenty for the day. It's generally easy to use. The main thing I watch out for is when handling the unit (taking it in/out of a small pocket on a haversack), the rocker switch might move the cursor on your map -- so before you hold down the switch to mark a waypoint, you may have to 'cancel' the cursor to bring you back to your exact location for marking. It's not inconvenient, it's just another one of those little mental checks each piece of equipment has when using/handling.