Hi Niels.
I did a similar sight-seeing and birding combination in May 2014, travelling cheaply and using mostly public transport. There are definitely plenty of birds to be seen around the cultural sites. Here are a few of my highlights:
Yerevan: If you visit the capital you'll almost certainly go up the Cascade - a huge set of sculpture-laden staircases climbing above the city and giving excellent views of Mt Ararat in Turkey. You can walk up the outside but there is also the option of a series of escalators inside (which I didn't find out about until I had walked all the way up three times in three days).
I had a Black-eared Wheatear on some rough ground at the side of the Cascade and, once at the top, the path skirts around a construction site that held Crag Martins.
Near the top is an obvious Soviet era monument and, by crossing the road to the east, you arrive at Victory Park, which I found to be the best city centre birding site:
A male Levant Sparrowhawk was seen briefly perched. A small scrubby area held a pair of Lesser Grey Shrikes together with a pair of Red-backed Shrikes, with two more RBS territories elsewhere in the park. Spotted Flycatchers and Nightingales were common and two Eastern Olivaceous Warblers were seen, as well as Common Whitethroats, Garden Warblers and Common Rosefinches.
Syrian Woodpeckers were seen here as well as in other city parks. Amongst other birds found easily in the capital were Laughing Dove, Hooded Crow, Hoopoe, White Wagtail, Turtle Dove, Hobby and Tree Sparrow. Four Honey Buzzards were seen on migration near the Hrazdan River and, just outside Yerevan, a Rufous Bush Robin was seen at the ruins of Zvartnots Cathedral.
Lake Sevan: Visited on a small group day tour around the lakeside monasteries. Armenian Gulls were common here, with around 300 breeding-plumaged adults seen. Other birds here included the black-capped krynicki sub-species of Jay, Black Redstarts, Rose-coloured Starlings, Cattle Egrets, Choughs, Alpine Swifts, Crested Larks and European Bee-eaters.
Mt Aragats: I stayed at the astronomical observatory in Byurakan, that has accommodation to rent, and the extensive grounds held Syrian Woodpecker, Golden Orioles (common), Nightingales, Scops Owls, Long-eared Owl (one heard only), Hoopoe, Lesser Grey Shrike, Eastern Olivaceous Warbler, Penduline Tit and Rock Sparrow.
From this base I did two trips by unofficial taxi (many drivers here act as taxis and will either stop for you along the road or someone from your accommodation can organise one). First I was driven to Amberd Fort and then walked all the way back, seeing Long-legged Buzzard, Bee-eater, Red-backed Shrike, Cuckoo, Ortolan, Rock Thrush, Crag Martin, Black-headed Bunting, Corn Bunting, Rock Bunting, Ring Ouzel (ssp. amicorum), Lesser Spotted Eagle, White-throated Robin, Rock Sparrow and Woodlark.
The second trip was a shorter visit to the lake on Mt Aragats where I saw most of the same species plus Twite (ssp. brevirostris), Alpine Accentor, Shorelark (ssp. penicillata) and Water Pipit (ssp. coutellii). Of course, both of these trips would be easier with your own car.
Dilijan: Very relaxed place surrounded by woodland. Green Warbler was easy to find here at walking distance from town. Also in the surrounding woods and at nearby monasteries were Black and Common Redstarts, Treecreeper, Grey Wagtail, Common Sandpiper, Raven and Tree Pipit. Caucasian Chiffchaff was heard around here but not seen very well.
Many of the birds mentioned were seen in multiple locations around the country. Other birds seen while travelling around included White Stork, White-winged Black Tern, Blue-cheeked Bee-eater, Roller, Egyptian Vulture, Griffon Vulture, Ruddy Shelduck, Dipper, Lesser Kestrel, Little Owl, Blue Rock Thrush, Stonechat and Yellow Wagtail (ssp. feldegg), as well as some species that are common in Northern Europe.
