Back Issue Article
Grist for the Mill: Objective Versus Subjective Audio Reviewing
Lets talk audio reviews. A recent exchange with another reviewer made me revisit certain issues that I first considered when I got into this demented writing-about-audio habit two years ago. Riding out ones inner thoughts through writing can do wonders. It brings vague concepts out into the open and in focus. It forces a writer to inspect the continuity of his thoughts, test the validity of his arguments and dig for overlooked counter-arguments. Afterwards, he might find hes won the rodeo, or that he got bucked off his wild-ass bronc in a quarter round, showing one and all how his grand philosophical ideas didnt hold water when confronted with the harsh realities of clinging to the saddle of reason.
Since youre about to read my short article, lets assume that I stayed on top of my beast long enough to smugly believe I had something useful to share. But youll be the judge. You can always bet on a different rider. That said, now take out your magnifying glasses, prop your squeaky-clean, silver-spurred and never-used-for-riding boots on the rail like a regular, and zero in yonder not on the horse!, but that gleaming new Q45 by Infiniti -- not the one on the road, mate, the one on the magazine cover that the foxy lady over there is studying instead of the ring. Pay attention! I said the car, not the girl.
Pick up any decent car magazine, regardless of its cover or whos reading it. Despite a certain unavoidable amount of emotional content -- after all, boys will we boys when it comes to toys -- youll notice how the bulk of automotive reviews remain not only very technical but also highly objective. How can you argue with a spec that clearly states how many feet it took a certain car to come to a full stop from 60mph? Or how many cubic feet of storage the folding up of the rear seats generated, and how many six packs of beer this equated in raw practical capacity when compared to a competitor?
Acceleration, top speed, quarter-mile performance, ground clearance, engine noise, tow capacity, crash safety, interior comfort, accessibility of controls, previous reliability records these and plenty of other performance attributes are readily assessed and agreed upon. Some, like comfort, cant be actually measured. Still, the degree of consensus amongst different testers is usually high. Thats because as long as they drive their test car over a previously agreed-upon type of terrain in the same weather, most drivers expect more or less exactly the same from the machine under evaluation. It either does the job or it doesnt. Can it take that steep left bank at 75mph, or are you forced to throttle down? Does it lose traction on a wet road from a dead stop? Do the seats retract far enough for a 6'3" driver to be comfortable? Are there interior rattles when crossing a cattle guard? And on it goes.
Suggest then the frivolous notion of subjective automobile reviewing to our ragtag team of test drivers. At best, youd probably be greeted with a round of hilarious belly laughs. More likely, youd find yourself quickly sized up with an incredulous glance from the team leader. Then the pros return to their previous discussion to leave you out in the cold feeling like a shriveled-up wallflower. For male-bonding fun, you might also be briefly accused for not having all your oars in the water, or asked what backwater planet you just escaped from. And should you really be surprised? A professional test driver for a reputable car magazine would cringe at the idea of approaching the review of a new ride from an emotionally subjective perspective. No doubt also hed get fired if he entertained such folly in front of his editor.
You already knew I wasnt headed for Le Mans, right? In audio reviews, there does exist a parallel tradition to the objective technical car review. Many such appeared in the late Audio magazine. Some of its key writers were not only fluently versed in all of audios intricate technicalities, but could back up their findings with fully stocked test lab. Audio's reviews would often feature five pages of technical descriptions anchored by incontrovertible hard-science measurements. And yeah, thered be some concluding listening impressions -- a few paragraphs, in fact, to the tune of "the amp performed like the measurements would suggest," or some other such tension-laden exclamation.
The trouble with this approach -- that is, trouble for most us readers -- arose from the concomitant proposition. You had to not only understand the significance of the published graphs and measurements, but also how to extrapolate what they actually meant in practical terms. What does an amplifier spec of 0.005% THD sound like? How does a speakers impulse response reflect on what itll do with music?
Thats a lot more mystifying than reading about a 0-60 acceleration in 3.9 seconds, yes?
You see the problem. Audio reviews such as these are essentially useless to the majority of music lovers. We dont understand the technical aspects of audio engineering. Honestly, we probably dont even care to know. No blame in that. Enjoy the meal and let the chef worry about mixing the proper ingredients. Besides, tech-heavy writing, like test scores, can be an awfully dry read and hard to stomach. Mind you though, if objective were your honestly stated objective, this and double-blind listening tests are the only type of reviews that can rightfully claim that crown. Objectivity requires repeatability of test results no matter whos behind the wheel. Can anyone with a straight face make that claim for audio reviews based on purely sighted listening impressions? Imagine four different reviewers hearing the same thing, then describing it in a way that you get theyre talking about the same thing. If youre not getting that, welcome to my lonely-hearts club!
However, there does lurk, in certain mags, a different kind of approach. It purports to be objective but really isnt -- when you apply the standards we just discussed. Driven by philosophical heft, garnished with arcane references to prior writings of colleagues, oneself or obscure recordings, a writer of that school fabricates the appearance that hes beyond subjectivity. His statements about even minutiae are absolute, and he loves to talk in the majestic plural: "The Giga amp, in its rendition of the upper violin harmonics, brought to mind the French Mega amp we recently put through its paces. In comparison, the French piece diminished what should have been a natural decay of tone, foreshortening it by a significant degree but retaining the proper blend of overtones. The Giga amp not only corrected this minor defect but added a very agreeable nuance of wooden body, such as youd hear during an inspired life performance in the Dorothy Chandler pavilion "
Beautiful, isnt it? Concise and factual, it leaves no doubt that this guy knows what hes talking about. He can compare, from memory, the finest of subtleties between components. Whats more, he uses as his reference live music. Whats even morer, he doesnt "seem to," "feel that," "perhaps," "it might," "apparently" -- no, he obviously tells it as it is, no ambiguity or subjectivity about it.
I dont know about you, but this type of approach causes an adverse reaction in my gut for a couple of reasons. The most important is the appearance of objectivity. He wants you to believe that here youre studying a technically sound and objective hardware review. You arent. These are nothing but personal listening impressions. Now look at the word, impressions. For the real whopper, go to "impressionist painters." Are their renditions photo-realistic? Obviously not. This doesnt belittle their beauty. It does betray any semblance of impersonal objectivity, though, regardless of the artists objection that "but this is what it looked like to me." It might have, buddy, but that was your personal perception. Nothing wrong with that as long as you say so.
My second criticism of our fictitious review excerpt is its pompous tone. Its plainly in the service of its fake objectivism, hoped to eliminate any doubts or second opinions, but it does rattle my chain. It doesnt talk to the reader, or better yet, with him -- it talks down. Hell, reviewing isnt rocket science or existential philosophy that it need be conducted with such air and timbre saturation of superiority.
Theres another type of review format that allows statements like "the Orgy 666 amplifier is slightly on the yin side of neutral and thus merits 87 out of 100 points for accuracy." A point system, though seemingly practical, once again suggests an astounding ability for precision on the part of the human operator. This is, frankly, beyond my facility to comprehend.
We all know that theres a very strong element of emotional response to music. Thats how music distinguishes itself from pure sound, after all. In fact, you could claim that outside of the human ear (and possibly those of certain pets) music exists only in the form of apparently random noise. If you took an Amazon pigmy to a classical symphony, he wouldnt relate. Rather, he might be terrified to death and attempt escape or cover his ears in agony. To appreciate and understand music requires cultural conditioning. Dont agree? Try classical Chinese opera for size. Youll complain about "noise" in no time.
Truly, the cognition of music as music is an acquired function light years beyond the ability of any machine. Otherwise, machines could compose music. From all this follows that anything other than machine-derived measurements and hard-fact explanations of said measurements involve a subjective response on part of the reviewer. It also follows that reviewer-listeners are endowed with different capacities to transpose the noise emitting from a system back into music thats governed by formal structures and harmonic laws. Im not an aural Darwinist to attempt an explanation for why these capacities differ. I feel certain that exposure for sure is a major factor. As an ex-musician, I can also vouch for varying degrees of innate talent that distinguish a superior artist from one who, despite hard work, will never gain that certain magic. Perhaps theres also a genetic connection then.
For arguments sake, lets posit that certain reviewers use pure test tones for review or manage to shut off their emotional/cognitive response during listening evaluations and transform into signal-sensing robots instead. Lets give them that. What good would any of their findings be, unless one were to listen to said components in the same state of turned-offness, either solely involved with test tones or not "translating" the noise back into music?
I say the moment an audio review uses music during its evaluation protocol, the component gets rightfully used in the same proper fashion as any owner would. Dont make my dream car crawl across the Rubicon trail if I would only use it as the blacktop cruiser it was designed to be. Dont subtract the emotional enjoyment from the reviewing procedure if you know perfectly well that all end users -- yes, in their own personal ways and to their own favorite type of tunes -- need to feel something to enjoy their music. A certain amount of subjectivism, some kind of involvement -- or the notation of lack thereof -- is unavoidable and necessary on the reviewers part to tell us something useful about the component. Does the component under review exhibit traits that interfere with the satisfying reconstruction of music, or does it invite them? Tell us about the components personality, its attributes and so forth, but dont forget that beneath all that lies something far more important.
This, of course, doesnt mean Im advocating irresponsible license to wax poetic. I simply feel strongly that no reviewer can rightfully pretend endowment with a superhuman ability for pure objectivism. Yes, he should attempt to subtract his personality to the best of his ability, to report on what seems quantifiable. Thats his job. But he should simultaneously remain cognizant that theres a hard-wired limit to how much he can stop being himself. Thus let his writings remain transparent to his knowing that!
In my ideal review, this is what happens. You understand clearly that the writer is making all efforts to remain objective. You simultaneously know that he admits that there are practical limitations to actually accomplishing this. Now your guard goes down, doesnt it? You realize that this reviewer fellow is just like you, sometimes not entirely clear on what hes hearing or how to describe it. With the guard down, you begin to relate. Instead of feeling preached to -- which, in the back of your mind, makes you wonder whether there isnt something remiss with your own hearing abilities, making you put yourself down -- you begin to recognize a part of yourself in the writing. Youll agree and disagree with certain segments, but youll be into it and getting something out of it. Youll also be reminded that theres only so much that can be said about audio in a meaningful way. Beyond that, its under-the-sheets time.
To accomplish this meaningful rapport with his readers, a reviewer needs to practice honesty and consistency. Its ultimately not that terribly important or even possible that his applications of less tangible terms like "air," "bloom," "transparency," "projection," "pace and rhythm" et al refer to exactly the same aural events that a colleague would describe likewise. If our writer always uses these terms for the exact same quality, in all of his reviews, the reader will shortly be empowered to feel on firm footing and relate successfully. This does require a certain discipline on the part of the writer, one that is purely self-imposed and self-monitored.
In an ideal world, all writers of a given magazine would attend joint seminars to learn how their observations overlapped or diverged from those of other experienced listeners. Part of such a dialing-in process would result in a standardization of precise and applicable terms to describe such observations. It would result in a streamlining or homogenization of the entire review process, to render the actual reviewers interchangeable, their review personalities submitted to the process rather than being allowed to play first fiddle.
In an ideal world, he said. Something about the homogenizing business rings less than ideal though, doesnt it? Sterile, perhaps? If thats your response -- and it certainly is mine -- lets admit that a certain artistic element in review writing is enjoyable and juicy. All great musicians play by the same rules -- a note is a note is a note, after all. If its there, you gotta play it, on time and with correct pitch. But we can also agree that their greatness lies in precisely how, and how much, they stretch and bend those rules to submit them to their personalities, rather than submitting their personalities to the rules. The notes an audio reviewer has to play with are his words, and whatever novel and meaningful ways he can invent to describe many of the essentially intangible and virtually impossible-to-pin-down elements of music reproduction. This frees writers like myself to look at our review fabrications more as creations of art and less as test scores. But unlike an artist, a reviewer is also a critic. (Would that most critics were artists as well!) The reviewer must sample the fare, divine its ingredients, point out the complexities of the mix, and then put it all into context. How does it compare? Is it worth the asking price? Does it set a new standard for whats possible?
From greenhorn to wiseass
This obviously requires experience. Going from bicycle to beat-up Oldsmobile in ones teens creates a different perspective on automobiles than having driven the lot by age 40. Some of the very established publications, on occasion, have used this very requirement to demerit the writings of up-and-coming reviewers. They havent seen the greater picture yet, they quip; let them first earn their spurs before theyll enter the races. How to earn them spurs without riding goes conveniently unmentioned. Ditto for how the accuser worked his way up from ground zero to become the kind of authority that can now pity and down-talk the newcomer.
Practice makes well, not perfect, but certainly more accomplished. Some mentoring helps, too. Thats often the job of the editor, who has enough personal experience to smell a rat when he sees one. Hell question the writer on certain statements when they seem misguided, poorly phrased or plain implausible. When he has reason to suspect a component mismatch, hell requisition a follow-up or sidebar review from another reviewer. He matches incoming equipment with the experience, exposure and interest level of his writers. Hes like a music teacher. He withholds those parts of the repertoire that require further technical or emotional maturity on the part of his pupil. In short, a good editor of an audio publication uses his own best judgment to decide who of his writers gets what when, and why.
In the course of such a monitored education, its only natural that each and every writer will eventually be introduced to components thatll ring his bell like a bat out of hell. They turn into his references to become examples of the best at a given price point. Such reactions are highly formative. They help develop what I call the audiophile persona. Had the dice fallen a different way, had a very differently performing but equally excellent piece come along first, our reviewer might have developed his preferences along very dissimilar lines. Theres no way of telling. Its good, though, to mind the consequences. It makes it clear how relative many statements, class ratings and recommendations really are -- and must be by their very design.
The checkered flags
And this brings us to the finish line of todays race around the audio review ring. Who won? Let me know. I look forward to your thoughts on the subject, whether youre a reader, manufacturer or reviewer yourself. Use the Industry Insider forum to communicate, and place "May Y-Files" into the subject header. Throw me something tough! Im German. Were gluttons for punishment. Until then. Cheers.
Copyright © 2001 SoundStage!
All Rights Reserved