I love the music of Taneyev and have (I think) almost every commercial recording released of his music as well as some broadcast recordings.
His "St John of Damascus" is arguably the most beautiful choral work ever written for Russian voices. I am not a huge fan or choral music, but this one grabs me every time.
The "Suite de Concert", op. 28, is a splendid violin concerto (in all but name)
The "Oresteia' overture is his most delicious shorter orchestral piece.
These are the three large scale works that I consider his best, other than the 4th symphony. They are available in many assorted couplings, which can make for confusing purchasing choices.
There are some outstanding recordings of the "Suite de Concert". Oistrakh was the dedicatee of the work, premiered it, and made the first recording, which is coupled with the Khachaturian Violin Concerto on an EMI Great Recordings of the Century CD. The same Oistrakh recording is coupled with Rostrapovich's exceptionally fine recording of Myaskovsky's beautiful Cello Concerto. The third recommendable CD is Ashkenazy's coupling of the "Suite de Concert" with the "Oresteia" overture and other orchestral extracts from that opera. This is a first rate recording of the violin piece, coupled with perhaps the best recording of the "Oresteia" overture -- in better audio than the other two. So, go for the coupling that interests you most.
With a deep sense of revulsion, I'll recommend the Pletnev recording of "John of Damascus." The alternative recordings are not as good, plus it is coupled with Rachmaninoff's magnificent choral symphony "The Bells." There are better recordings of "The Bells", but if you do not have it in your collection, this is a good coupling.
If you are as disgusted as I am by Pletnev, I'd say to go with Sanderling. I really do not like Fedoseyev's approach to music at all, Polyansky is kinda mushy sounding, and Golovanov is eccentric.
If you do not go with Ashkenazy's recording of the "Oresteia" overture, the next best recording is by Jarvi, coupled with a very good recording of Taneyev's 4th symphony. You already have the 4th, but if you do not have Jarvi's recording, you might decide it is worth the investment. Very good audio.
As for the chamber music, the big chamber music competition/festival in Russia is rightfully named after Taneyev. Although some of the string quartets can be a bit dry, I really like the Piano Trio and the Piano Quartet. There are several fine recordings of the Piano Trio. I like the Borodin Trio's recording, but there is no coupling. The Taneyev Quartet coupled the Piano Trio with the Trio for Two Violins and Viola, an unusual instrumentation. The couple of both the Piano Trio and the Piano Quartet can be had on a CD by the Mendelssohn Piano Trio.
Once again, if your disgust permits, the recording of the Piano Trio coupled with the Piano Quintet is well performed.
Discovery of the music of Taneyev was one of my great unsung adventures, and his music remains among my favorite of the unsungs.