Don’t Confuse Experience with Expertise or Evidence

In May, I was speaking at a Summit in Pensacola, Florida. A large group had gathered from Northwest Florida to hear about the state’s work in ending homelessness and practices in ending homelessness. I love these sorts of engagements, especially when I am given half a day to walk people through the critical details rather than just hitting the high points as I may do in a keynote or media interview.

In the morning session, I was absolutely astounded by Volunteers of America. Staff were there to speak about securing funding for operations and sustainability of those sources. To start, however, they got into talking about their approach to supporting people in housing. According to them, they will do whatever it takes to support a person to get out of homelessness forever, but that housing first does not work. Grant and per diem programs were, apparently critical for some people, as were shelters or other types of transitional programs for others. When I asked, “Where is the evidence?” there wasn’t any. When they touted the amazing success of their approach compared to all other approaches I asked about control groups and comparison studies. To paraphrase what I put on Twitter, it was amongst the worst opinion-laden crap that they were putting forth as fact that I had ever heard.

Unknown

Two things then happened that were not a surprise to me. For one, the speaker had no evidence. Zip. Zilch. Nada. But he did call upon other Volunteers of Americas staff in the room to share stories and anecdotes. The second thing? The speaker and other Volunteers of America staff in the room then talked about how “in all of their years of experience” they know what works and what doesn’t, and then reached to communicate with the crowd that they too just know what works and what doesn’t…that housing first and rapid rehousing may make for good academic studies but does not work on the ground.

There was zero proof that any of their ideas were rooted in fact. There was zero proof that any evaluation had ever actually occurred. There was zero capacity to meaningfully answer the questions they were asked to provide evidence to support their ideas. And that is a problem. Especially when they were there to talk about how they brings in gobs of money in Florida and nationally.

How does an organization as large as Volunteers of America get as big as they get, with the volume of funding they get in communities throughout the United States? I am not so naive as to think the small sample size of Volunteers of America staff speaking in Pensacola are indicative of the entire organization, even though they made claims that theirs was the position of the entire organization. I think the bigger issue is that we (the collective we…funders, government, continua, other service providers, etc.) continue to confuse experience with evidence…that just because a person or organization has been around a long time that must mean they are effective and qualified.

And when a large organization has information that they are sharing challenged publicly? Well, never underestimate an organization’s desire for self-preservation. They doubled down. They indicated they could not stay for my session in the afternoon which talked about theory, practice and EVIDENCE of how to operate effective housing programs. The reason? They had a plane to catch. Which, like they said during their morning session was not true. How do I know? They were at the airport later in the afternoon waiting for their flight at the exact same time I was waiting for my flight. Oops on their part. Here is proof:

IMG_2542

Iain De Jong

About Iain De Jong

2 Responses to “Don’t Confuse Experience with Expertise or Evidence”

Read below or add a comment...

  1. udjibbom says:

    It’s also possible that people with day-to-day, on-the-ground experience have valuable insights and perspectives that are worth considering. I don’t know anything about this particular group of Volunteer for America representatives but, just going out on a limb here, maybe they have a background which doesn’t include statistical data collection or analysis – maybe (shot in the dark, I know) they’re social workers who are busy serving client needs and only collect the data they’re told they must, in order to continue doing what they consider to be the real work? (that would be helping people end their homelessness, not flying around the world giving lectures.)

    You’re the stats and data guy, Iain – why don’t you take a look at their data and see if there is any evidence to support their assertion that housing first or rapid re-housing programs aren’t working in their area? Was anyone in this group told they would need to produce evidence or bring data? If not, then it is completely unfair for you to dismiss everything they told you as nothing but stories and anecdotes, to say nothing of using your literal bully pulpit, microphone and status to belittle and denigrate them..

    You could also look at the level (sorry – “gobs”) of funding they’ve received and the housing costs in their area, to see if the funding they’ve managed to secure is sufficient to serve people or not. At the very least, you could offer to donate some of your time to work with this group to give them some background on the data they need to collect or teach them some techniques for how to interpret the data they already have. Maybe the evidence you’re looking for is just sitting there on the HMIS, inaccessible because their HMIS vendor hasn’t figured out how to report on data like recidivism…

    You could also use your position as an industry leader to advocate for more funding for PAID case management time and services (not volunteers!), so the people working on the ground actually have the luxury of a few hours to consider the data and think strategically, not to mention put together the evidence you consider so important, instead of having to chose between looking at numbers on a page or serving the real, actual human being waiting outside their office right now.

    But we get it: it’s easier to post a couple smart-ass tweets and blogs instead of doing something difficult. Believe us, we know.

  2. carole says:

    I often hear in my community that approches from other communities won’t work here in Sudbury. I think thats bullshit, our community has no evidence to suggest weather it works or doesn`t cuz evidence has never been collected and we haven`t delivered this type of service. Our city adopted the housing first strategy along time ago but never implemented it, I don`t think the people in charge know how to be honest! I believe in housing first, and applied the same perspective and I know it works.