So who do I trust to bring me independent and objective views; where do I get information from about the products out there? I’ve been asked this question recently and there’s also been some mutterings on Twitter about the lack of transparency about some of the things out there.
I get my information from a variety of sources as I suspect we all do
Journalists: Now, I must lament some of the technology journalism; it’s sheer churnalism, the regurgitating and almost copy and pasting of press releases. I might as well go direct to the PR agency or the vendor. But we do have the odd good one who do try to hunt down a story, not take the vendor’s word on everything; yes, Mr Mellor, I’m thinking of you.
But even the churnalists, I generally trust that they are not being paid directly by the vendors with the exclusion of advertising revenue. I trust that you are not spinning a story to make the vendor look better. This may be naive but I hope this is true.
Bloggers: Well there are regrettably few independents left in the Storage World, it is nice to have a voice which is both speaking from experience but also from neutrality. Yes at times, I question whether anyone can be truly neutral and the FTC rules around disclosure are pretty good in allowing me to make a judgement. I think we have to be careful that we don’t become the ‘tech liggers’, turning up at every tech event on expenses paid for by a sponsoring vendor.
This does make life a bit hard, certainly for those of us who have corporate jobs but I think in general we do pretty well at managing this.
Peers: Probably the most useful of all my information sources, swapping experiences and opinions about technology. Sometimes having the conversation which the vendor really does not want you to have and puncturing the unreality bubble surrounding some technologies is incredibly useful. It would be great if we could have these conversations more publicly but we have families to feed. By the way, having worked the other side of the fence; I know the conversations about customers and shared experiences there could be equally career limiting if made public.
Analysts: Unfortunately I have to the stage where I believe almost nothing an analyst writes unless somewhere in the piece it discloses who paid for the piece. If there is no disclosure, I just don’t trust it; the analyst companies such as Gartner have been so opaque about their dealings with vendors that none of you are trusted.
And you really don’t help yourselves, lets take ‘The Cube’ for example; how many people who watch those videos realise that it’s a pay to play thing? They are generally flagged as interviews but you need to flag-up that these are paid for slots. So Wikibon who were supposed be a new form of analyst find themselves in that mire.
I hear conflicting stories about all analyst companies so it is probably unfair to pick on Wikibon and Gartner specifically but I think that all analyst companies should have a piece on their Business Model and how they make money. If you are taking money from both vendors and subscribers, I would like a disclosure of the proportion of your revenues are from each sector.
Squeaky clean is what you have to be.
Vendors: I trust all of the information I get from vendors….to show their product in the best possible light at the expense of the competition. I expect praise for competitors to be faint at the very best. But I do get good information from vendors and if you filter well you at times get the odd nugget about a competitor.
If a vendor tells me there’s an issue with another vendor’s technology; I generally will ask and often there is a grain of truth. Sometimes it’s an historical thing, sometimes not but it’s a whilst since I had a vendor tell me a blatant untruth. Of course I much prefer vendors to talk about their own stuff.
But when vendors get my goat is when they selectively quote from a piece which shows them in a good light and when you dig, you find it’s a piece that they paid for or with a company that they have a commercial relationship with.
And don’t be weasels and hide the relationship in small print; no-one reads small print.
Great write up, hope I am not a weasel…. #;-)
The situ with analysts interests me… I suspect clients simply are not willing to pay enough to sustain a decent (and well paid) analyst community (everyone wants everything for free). How to create a sustainable analyst business model that does not get compromised by ‘closeness’ is something the auditing firms have pretty well failed at….
I think as long as everything is transparent, it can work. When it is opaque and there are whispers about said opaqueness; it breeds a huge feeling of mistrust.
Transparency should be simple and not scary regardless of if you are an analyst, advisor, consultant, blogger, journalist, var, vendor, customer or what ever as long as you disclose and are not playing games that need to be hidden. On the other hand, if games are being played, chances are that there are games being played with the disclosures.
Thus you will find disclosures on StorageIO related content.
Hope all is well
Cheers
gs
Disclosure – EMCer here.
‘bod – thanks, good piece. I truly shrink at the thought of where I (personally) would fit in your view. NO – don’t tell me. The thought will always help me try to tread as carefully as I can.
For what it’s worth, I’m taking the discussion about transparency as seriously as I can. Look the fact that I work for EMC is a fact. Wikibon (and just about every analyst) has, or are, done paid for pieces for us. They can provide value – but I think you (and Knieriemen too) are right on – there needs to be total transparency. Chuck’s on the same wavelength internally too (and publicly).
A guy trying his darndest to do the right thing always (but like all humans occasionally failing),
Chad Sakac
The best scenario would be some way for end users to exchange information with each other on an internet scale. While this happens to a degree on the open internet, its hard to know what sort of person you are dealing with, unless you happen to engage the community on a regular basis. That’s like having an additional part time job. It can be intimidating to start opining openly about vendors and products – and as many have found, the “web” is a complicated place with plenty of surprising personal risks.
Wikibon started out trying to find a way to connect customers together in a meaningful way in an online community and I give them a lot of credit for making a hard run at it. That’s where the “wiki” in the name comes from. Dave Vellante, David Floyer and Peter Burriss (perhaps the most talented technology writer I’ve met) had a very good concept but couldn’t make it work as a business. Its just too difficult finding enough end users with the time and motivation to do it – and are willing to do it within the confines of somebody else’s community site. It’s much more appealing to just start a blog or a twitter account.
Wikibon had to change their business model to survive and became a company that is more or less like other analysts. I think Dave, David, Stu and the team there are good technology thinkers and I usually learn useful things or get a good alternative perspective from them. There are a lot of good tech writers in the analyst business and I could go on and name my favorites, but that would get weird.
An under appreciated part of this business is the nature of the relationships between analysts and the vendors who buy their services. A successful analyst company has a lot of clients – including many competitors. They have to figure out how to walk the line among competitors that are “emotionally estranged”. Storagebod wrote about customers swapping stories about vendors (and vice-versa) but this also goes on to a lesser degree among analysts who occasionally will compare notes on certain quirky, abusive or otherwise “interesting” clients. Sometimes these people are the CEOs of large companies. The point is – it isn’t as much about the money sometimes as it is the politics and pressure and those sorts of disclosures just aren’t going to happen.
All news and information is interpreted, written, edited and produced. We like to think there is a free press, but that doesn’t mean the news can’t be distorted by political bias or other self-serving interests. As Storagebod sort of suggests, the best way to interpret what an analyst says is to read multiple pieces of their work and try to pick up on their tendencies and biases. Of course, that means setting your own biases aside so you can be objective in your own judgments. It’s easier said than done.
I’m Marc Farley, I work at HP and am a producer/co-host @Infosmack
I suppose I should start by thanking you Martin. While I know it’s cool to hammer Gartner I personally know several analysts at the firm and have a great deal of respect for many of them. Gartner is a firm that completely changed the research and advisory business, which is what we set out to do at Wikibon; so I guess I’m flattered that we’re being mentioned in the same paragraph.
But I can’t think of a good reason why you’d make such statements about theCUBE and Wikibon without contacting us first.
Our partner SiliconANGLE has always disclosed when it has sponsors for theCUBE. TheCUBE is a live event, much like an O’Reilly Media or GigaOm conference. SiliconANGLE is constantly disclosing its sponsors. As an example, here’s the advert for the upcoming VMworld event that clearly lists the sponsored segments:
http://wikibon.org/blog/wp-content/uploads/2011/08/vmworld-live-2011.html
Are you looking for something more to meet your transparency criterion?
Reasonable people can debate the quality, format and independence of theCUBE segments (which I’d be happy to do) but your issue seems to be disclosure– which clearly SiliconANGLE has always provided despite your assertions to the contrary.
As it relates to Wikibon – we have nothing to hide. Here’s an excerpt from our FAQ on how we make money – I’d be happy to make improvements if you have suggestions. I would respectfully suggest that rather than taking the so-called ‘whispers’ as fact, you owe it to your readership to do a little bit of homework before taking cheap shots.
Wikibon’s content is open source and freely available to anyone. We derive our income from three primary sources:
1. Statement of Work-based consulting services. Our roots are in technology consulting. Our founders have deep relationships with IT execs and executives at tech companies. Over the past several years, technology buyers and IT suppliers have paid us for detailed research, surveys, cost models, architectural roadmaps, portfolio assessments and advisory support.
2. Data. We integrate and package data and information products based on knowledge derived from the Wikibon community.
3. Media partnerships. We provide research and content services to media organizations, which they monetize through advertising. We participate in a revenue share with such organizations.
The community prides itself on independent analysis and as such, we strongly encourage that works published on Wikibon are not to be reviewed in advance by an author’s clients or sponsors. Wikibon employees are not allowed to produce commissioned white papers and by policy are not allowed to share works with clients prior to publication. We write for a user audience. Because Wikibon is an open source community, anyone can use it as a publishing platform and there have been instances where paid for pieces have hit the site. However all contributed works are edited mercilessly and open to peer review. Specifically, anyone in the community is free to edit the pieces and we strive to produce research that is peer-based and accurate. If someone disagrees with our analysis, they are encouraged to hit the edit key.
Cumulatively, since our inception, about 20% of our revenue has been derived from end user organizations, 60% from technology suppliers and the balance from financial institutions, media partners, investors and other sources.
David,
maybe I am wrong to pick on Wikibon but I’ve had a number of conversations with my peers, end-user peers and there is not a lot of trust about. But that might be a distrust of analysts in general and as the new guys on the block, you are just picking up more than your fair share.
I’ve pretty much given up on the analysts with few exceptions; I’d rather read a vendor blog as opposed to most analysts. At least I know exactly who is paying for the words, I’m not at all sure with many analysts.
Maybe I’m just cynical….but there’s a lot of us cynical people out there. Maybe it’s British thing, not sure..
Martin
Now who is being opaque Martin? You call out the analyst community generally and theCUBE and Wikibon specifically for lack of transparency and when I share clear examples of our disclosures you ignore/dismiss them and continue to defend your premise based on ‘whispers’ and generalizations. It’s quite disturbing really. My colleague David Floyer who is also British would frankly be shocked at such anti-British behaviour (notice the excellent spelling 🙂
I have hundreds of specific examples of end user organizations that trust, appreciate and are thankful of our mission at Wikibon. My colleages and I facilitate many inquiries from IT professionals looking for help and there is no issue of trust. They know and respect our process and analysis. They do their homework and form their own opinions and we often are a very helpful piece of that decision-making process.
What you seem to be missing is the fact that we agree that peer-based information is by far the most valuable. That’s what Wikibon is all about and one of the things that makes us different. Everything we produce is editable so if you don’t like our conclusions and analysis – hit edit. We welcome all respectful contributions.
Yes we find ways to make money, to live, support our families and grow our community. We’ve always been open about how we do that. My question remains – since you specifically called out theCUBE and Wikibon; what are we doing that is objectionable to you and your unnamed peers?
Please be specific so we can honestly make improvements if we’re falling short of acceptable standards.
Okay, I actually asked a specific question of a Vendor with regards to the Cube whether it was pay-to-play and got an affirmative, along with a comment that they thought everyone knew that. I’d never really thought about it really but then again, I’m not a big fan of the Cube but I did think that they were honest-to-goodness straight interviews.
Now obviously, you are now telling me that they aren’t pay-to-play. So bit of a dilemma really.
To be honest, in hindsight; I probably might not have bothered calling out Wikibon and Gartner specifically and it was probably not wise. Mea Culpa. But I did so because I think there is little difference between the old-school and the new-school analysts.
There’s not enough snark, not enough pointing at ‘Emperors wearing no clothes’, a fear that if you are too critical then perhaps you won’t get paid in future.
But Wikibon specifically, where you are getting involved with mashing-up media and working with paying customers…then it almost has to be big flashing words flagging it. And I don’t think it is always clear IMO.
Personally, I think that I think it is sad that Wikibon takes vendor coin…I think it makes it incredibly hard for it to be what it aspires to be and hence you end up in the pot with the rest of them.
theCube which I run as part of my blog siliconangle.com is pure editorial. It’s ok to criticize what you don’t understand and that’s ok for you and your opinion. Trust is earned and your accusations make you look like a tool. As they say to a hammer everything looks like a nail. Everything is transparent you’re just stuck in the past.
Happy to debate and discuss with you anytime.
theCube is not pay to play at all pal.
John, so why I am hearing from vendors that they thought that everyone knew that the Cube was pay to play? Why do I hear from vendors that they thought that everyone knew that Gartner was pay to play?
Why do I hear these things? Why is it even possible? Because quite simply the analyst community, both old and new have not been transparent in their dealings.
As for walking the talk? Ummm, I take no money from vendors for this blog. I do it because I care about the community and Enterprise IT. I do it because I enjoy it but on a cost/benefit analysis…it’s a stupid waste of time for me to do.
Here is the SiliconANGLE.tv productions agenda for VMworld 2011…Thanks to EMC, NetApp, HP, Intel, VMware, Soldfire, Broacade.. recognizing excellence in action…
70 guests; 19 panels; VCs, Startups, Executives… I’m proud of this program.. Who pays for this … I do some support from sponsors disclosed… trust is earned.. I suggest you walk your talk pal.
http://siliconangle.com/blog/2011/08/25/announcing-siliconangle-tv-vmworld-live-thecube-schedule-8-hrsday-x-4-days-of-live-video-coverage-of-vmworld-2011/
[…] of us who have corporate jobs but I think in general we do pretty well at managing this. Read on here Advertisement GA_googleAddAttr("AdOpt", "1"); GA_googleAddAttr("Origin", "other"); […]
I think John Furrier addressed the ‘pay to play’ issue clearly in his post. Again – who is being opaque Martin. You have some mystery conversation with some unknown vendor and you claim to be the authority on the model.
Not enough snark? Not enough pointing at ’emperors wearing no clothes?’ I just went to the Wikibon site and saw this post from a member– full disclosure – HP is a sponsor:
http://wikibon.org/wiki/v/Finding_Focus%3A_The_New_New_Strategy_To_Get_HP_Back_On_Track
First it’s lack of disclosure and now it’s something else. It’s clear you have another agenda here.
As I always encourage people – if you don’t like what you read – just hit edit. Make a contribution to the community and see if it holds up to peer review – that’s the ultimate authority imo.
Martin:
Your blog post is spot on and you can tell that you hit a nerve. Let’s parse some of their comments…
Dave Vellante: “Our partner SiliconANGLE has always disclosed when it has sponsors for theCUBE.”
Furrier: “theCube which I run as part of my blog siliconangle.com is pure editorial”
Completely untrue. At last year’s VMworld, vendors were offered premium spots on The Cube in exchange for a very generous “retainer” to Wikibon. YET… during those streamed interviews, the viewers were not told that those spots were bought by the vendor, no disclosure, nothing. Simply ask Vellante and Furrier to post a link to any video from last year’s VMworld that had a disclosure on the video.
Dave Vellante: “As it relates to Wikibon – we have nothing to hide.”
Really? Take the recent Wikibon blog post “EMC Bleeds Only One Color…And That’s Green” (http://wikibon.org/blog/emc-bleeds-only-one-color%E2%80%A6and-that%E2%80%99s-green/). Can you find the disclosure on that blog that EMC has retained Wikibon? Nope.
What about supportive “editorial” articles of vendors like EMC on SiliconAngle.com? Any disclosure that those same vendors paid Wikibon/SiliconAngle for other services? Nope.
Now I challenge you Martin to watch the flaming responses you’ll get now from the SiliconAngle.com and Wikibon gang… Will they respond thoughtfully with specific examples of disclosure where I claim they have none or will they make personal attacks, question motives and do everything possible but address the specific examples I have cited?
Their response will tell you everything you need to know.
Mr Greg: you are incorrect again. Please stop with the slander. No one paid for premium slots – that is the fact. Where we have sponsors we disclose them see http://www.ServicesANGLE.com.
Content program for VMworld is impressive and that is what I’m focused on. Please stop slandering. Move on.
http://siliconangle.com/blog/2011/08/25/announcing-siliconangle-tv-vmworld-live-thecube-schedule-8-hrsday-x-4-days-of-live-video-coverage-of-vmworld-2011/
Martin: I did use your “churnalism” line today..that was good.
Like many things…the churnalism line is not original; I got it from Nick Davies out of his book ‘Flat Earth News’; it’s worth reading, especially if you want an insight into how journalism has deteriorated over the decades.
I started out as a consultant and soon started writing for trade magazines, you remember them they were 400 pages of paper that came in post. Back when there was enough ad money to pay for writers and labs for us to test stuff in there was no worry about disclosure because we writers were absolutely forbidden to take anything of value, including travel expenses to say a user conference like HP Discover, from companies we wrote about.
I had a consulting deal with Seagate Software for a couple of years before they were devoured by Veritas and didn’t write anything about Backup Exec or the rest of their products for another year after it ended.
Even so we were regularly accused of writing nice things about advertisers and leaving out others. At PC we did all inclusive bake offs (100 modems) and there advertising would improve your chances of inclusion, but not of a good review. We did that because would be embarrassing to run an article titled “we test every router” next to an ad for a router we didn’t test.
Now things have changed a little and I wear more hats. Since ad sales can’t support test labs but I like testing products to see if they work I work as an analyst at DeepStorage.net. There we do sponsored research and testing and every document that goes out under the DeepStorage banner says who paid for it.
I have some ideas about some broader research, like a survey of enterprise SSD systems, if I get to publish that sort of thing it will say “X, Y and Z have hired DeepStorage in the past but had no direct input into this report”.
I write at Network Computing as a journalist. I write about what I think is important period. If I mention a company that’s paid me for work in the past 2 years it’s noted at a clear disclosure in a separate paragraph. As I know a glowing blog post about a client’s product may not be taken as seriously as it should I occasionally don’t write what I thought would be a good piece.
All our work for vendors is piece work. We don’t have annual subscriptions. We do ghost write some white papers for clients that go out under their names. These of course don’t need disclosure as they say they’re from vendor X that paid for them. We will disclose them as a client if we write about them in other work.
I think analysts that work on a vendor subscription basis have an really fine line to walk. Does vendor X pay my rent? When I write about vendor X’s market, even if I am trying to be neutral, does that merit disclosure? After all I have definitely been briefed and have access to my clients and may not be as knowledgeable about their competitors.
Those are questions Greg has an easy answer to but if ESG or Wikibon had to list their whole client list it would put them in a difficult position.
I know some analysts will take a subscription or consulting fee and then do a blog post or the like that wasn’t directly paid for by the vendor without disclosure. After all they didn’t pay for that.
Howard, you said “if ESG or Wikibon had to list their whole client list it would put them in a difficult position.”
Really? A simple footnote in an article or blog about a specific vendor that says “(Vendor) is a client of (analyst firm)” is difficult? Is it more difficult than losing the trust of your reader?
Firstly, for any conspiracy theorists out there….I’m not working with Greg to bring Wikibon down or anyone else out there. I don’t know Greg well enough to work on such a plan and to be honest, I have no motivation to do so. I don’t plan to start a competing business and although at times the some of the UK #storagebeers regulars have had slightly tipsy conversations of doing something similar, we are much too lazy and disorganised to do so.
If I published the mails and DMs I’ve got; mostly supportive, it would show that there is still dissatisfaction with the analyst community. Many vendors and end-users a like worry about transparency but they still continue to use them.
Perhaps it is hard for an analyst company to publish their client list but I like the suggestion that when publishing a piece; even if it is editorial, that if an analyst has been on the clock in last two years, then it should be disclosed! Does that dilute your message, not as much as suspicion that you might be trying to hide something.
If a webcast, podcast or whatever has been sponsored by a vendor; it should be disclosed, preferably in the opening comments somehow or at the very least by a disclaimer at the end.
In an ideal world, we would have an analyst ‘Record of Interests’ detailing what companies have paid them in the past. In a really ideal world, the amounts paid…but I can see that being problematic.
Greg,
If I for example write an article that says “Emulex has better HBA management than Qlogic under ESXi” I would feel obligated to put “Emulex is a client” even though they pay me for completely different services.
If I were bigger and wrote “Thoughts on the SSD only storage market” where I mentioned 15 companies and 7 were clients it might get dicey.
So for the specifics of your question I agree. How far to extrapolate is an issue.
This thread is heading in one direction. But this is what I prepared earlier.
Martin’s post is about a number of things: trust, sources of advice, transparency and disclosure.
There is also the issue about the various hats people may wear. I communicate with numerous people who fit several of the job functions mentioned. Anybody can blog & tweet and many do, which makes it important to understand people’s motives and who pays whom for what. Bloggosphere went from being a group of enthusiastic volunteers, and eventually shuddered when it emerged that some could make an income from this activity. It underlined that people are not pigeon holed, even though life was simpler before and more innocent.
While one group is searching for truth, then another is incentivised to hush things up. Both are aspects of human nature and not necessarily evil. The best source of information and honest insight is peer-to-peer communication, however that content is often not in the public domain. I even heard discussions about a ‘wikileaks’ for the IT industry. I predict that that will be short lived, were it ever to be attempted.
So I understand Martin’s frustration and his analysis by job type. But, as mentioned people operate with more than one hat on. I make a living in the analyst category and Martin tells us that this group is earning the least amount of trust. Being in the employ of IT vendors does not inherently equate to spewing out deceitful kow-towing analysis. But trust is indeed earned. By being a reliable source of insight and good analysis is the best/only way to earn confidence.
Scepticism is healthy and demanding transparency is good. And the world is complicated for good reasons. Customers sometimes choose to be reference accounts. I bet other IT professionals take those endorsements with a big dose of salt.