By Pradeep Ramachandran
MSU recently released their 2017 codec comparison, where they compared x265 against other HEVC encoders, VP9 encoders, and the AV1 encoder. While the efforts that go into such large scale tests are appreciated, we as the x265 developers have to respectfully disagree with your conclusions drawn from this MSU report as we believe it is incomplete.
If you notice, these tests use v1.9 of x265 which is over 20months old! Since then, x265 has had 7 versions with an imminent 8th version. As anyone would expect, x265 has made considerable progress in speed and quality during this time. Specifically, we’ve made big changes to the lambda tables which considerably improved visual quality as reported by both consumers and customers.
That said, it is possible that maybe AV1 is a better codec than HEVC (at least in quality); maybe so is VP9. Maybe they have tools that are competent and can challenge HEVC. However, for the above said reasons, these results do not conclusively prove so, in our opinion!
Now as to why MSU used a 20month old encoder, there is some history there about the reservations of the x265 team to the validity of MSUs past tests as only objective scores were looked at. Quoting my dear friend, ‘we look at video and not at graphs!’. It is encouraging to see the MSU tests taking a turn towards doing subjective testing, which, in our opinion, is the right direction. Hopefully we will work with their newly mended ways in future for a more fair and realistic evaluation!