Formative Evaluations

My last project at the DIA was just as fun and interesting as everything else I had been working on all summer, and I'm a little disappointed that I didn't have more time to get involved with it.

Museum evaluation is actually very interesting to me.  I like to know how the visitors perceive the museum-- is it meeting their needs?  Their expectations?  Are they enjoying themselves?  Are they actually learning anything?  Is our message being received by the public?  And I find evaluation to be tied very closely to education, as the answers to the above questions are generally addressed by an educator.

The DIA appears to take evaluation very seriously.  They have their own evaluation department that works with many areas of the museum (staff engagement surveys to membership evaluations and visitor surveys). 

My last project involved formative evaluations of some labels for an upcoming exhibition called, "Fakes, Forgeries, and Mysteries".  I wish I were going to be in town for it-- it sounds pretty cool. 

The DIA actually tests every label before it goes up on the wall.  More specifically, they test interpretive labels and texts.  Those labels that just describe the artwork don't really change.  I love this.  I love the idea of presenting the draft of the label to the public, asking if the text works for them.  It just makes so much sense.  And honestly, it's not that much work.  I know it sounds really labor intensive to test every single label, but it's just not that bad.

All of the necessary supplies for label evaluations

Each draft label is printed out on regular paper and taped to a wall in the museum (frankly, in the scope of this entire project, I was most nervous about taping things to the walls of the DIA) and then 10 visitors are asked to read the label(s) and answer some questions about what they read.  Generally, 2 labels are tested each time so it goes pretty quickly.

The draft label on the wall, along with sample artworks

For two days, I was charged with standing near the Rivera Court, temporarily mounting the draft labels, and interviewing 10 people per day about their thoughts on the texts.  I really liked it.

To some, I'm sure this sounds like a painful task, but I really enjoyed for a couple of reasons.  First, I got to talk to visitors.  I love talking to visitors.  A buddy and colleague of mine, who works at another museum said, "I take my employee badge off when I walk through the galleries so no one will ask me anything".  We could not be more different in that respect.  In his defense, he's only been working at his current museum for a few months, so he doesn't really have the knowledge base to feel comfortable answering visitors' questions.  I, on the other hand, have been coming to the DIA for the better part of my lifetime and can tell you where the bathrooms are without blinking.

As a "prize" for participating, each visitor gets a postcard of an artwork in the DIA.

One of my "hobbies" while at the DIA this summer, was to walk around the galleries when I had some down time, and listen to people's conversations.  It sounds creepy, but I'm interested in what they are saying about the art and about the museum.  I like listening to how adults talk about art with kids, and how people engage with each other in conversations about art.

So it was a real treat for me to stand in the hallway and ask visitors questions.  Plus, I was able to give directions to elevators and bathrooms all day.

The other thing I liked about this project was seeing the results.  Because each label is only read and evaluated by 10 people, it's possible to see the results very quickly.  For example, after about 6 interviews, I was able to find a pattern and see that people had trouble understanding the third paragraph of a certain label. 

It's instant evaluation gratification and I loved it.

All Staff Meeting

Today I had to be at the museum before 9:00.  I kind of hate being anywhere before 9:00 but it turned out to be well worth the trouble.  It was the All Staff Meeting!

More than 100 DIA employees gathered in the auditorium (which I had never seen before, but is very nice!) to hear the Director and various other managers discuss projects and initiatives within the museum.  I asked how often All Staff Meetings occur and was told that they try to have them quarterly, but that's not always the case.  Either way, I was pleased to be present for this one.


Perimeter Heating Project: Elliott, Museum Operations

We are now in Phase II of the perimeter heating project (I must have missed Phase I), which involved closing a number of galleries on the second floor for repairs to the heating system. Last winter, some galleries with exterior walls had NO heat beyond the forced air system (I don't know enough about heating and cooling to tell you why we need more than forced air heat, but apparently we do). Fortunately, last winter was mild enough that there were few to no complaints from visitors, and of course, the artwork was unharmed.

Elliott, from Operations, attempted to list the second floor galleries that would be closing but admitted to not knowing the official gallery numbers. At this point, he asked for some “audience participation”, saying that he would tell us what is in each gallery, and someone in the audience should yell out the gallery number. It went something like this:

“It’s that gallery with the painting on the ceiling…”
“W 234!”
“It’s got that piece of furniture with all the inlaid stone… umm… it also has The Wedding Dance…”
“W 230!!” “BINGO!!
Ok, nobody yelled out “Bingo”, but I think it would have been appropriate.

After Gallery Number Bingo, Director Graham Beal talked about some stuff that was not on the agenda, but proved to be pretty interesting.


New AAMD Environmental Standards: Graham Beal, Director

The AAMD has decided to relax its environmental standards for galleries. The gold standard for relative humidity levels in the galleries was generally between 40% to 50%. Yet, for various reasons, the range of acceptable RH has been expanded to 40% to 60%.

 I did some quick Internet research and it seems like most museums were already doing this anyway. But now the AAMD is making this the official standard, which really only changes one thing; loans. As Graham explained it, some museums were having trouble acquiring loans because when they would submit a facilities report, their hygrothermograph output would read 56% or something that the loaning institution would find unacceptable. Graham even told stories of institutions submitting a blank hygrothermograph output sheet with a ruled pencil line drawn through the whole thing at 45%. In other words, some museums, desperate for loans, would forge their RH reports. Seriously?

The other benefits of this greater flexibility are that it saves the museum some money, reduces energy consumption and carbon footprints, and generally streamlines the loan process (so people can stop lying!).

Apparently it has been suggested for a while that most works of art will not sustain damage from incremental RH fluctuations, and can thus withstand a greater range. My peers that watched “The Rape of Europa” with me last semester can attest that many works of art that hung out in caves, barns, and other locations without climate control for the duration of WWII were returned to the museums without much (or any) damage. I am not suggesting that galleries be converted to reflect the barn environment in the name of reducing our carbon footprint, but I can certainly see the merit to allowing RH to reach 60% +/- 3.

Graham also noted that it is extremely difficult to retain a consistent RH % in the Midwest (especially in those dry winter months). So this change in the standards will end up saving the DIA quite a bit of money. In the winter the DIA will heat the museum less, thus reducing the need to add humidity to the air; and in the summer we will cool the museum less, thus reducing the need to remove humidity from the air.

During this briefing, a representative from the conservation or registration department stood up and assured the crowd that the collections were not being put at risk in any way by this change and that there simply won’t be much difference to the state collections.

To prove that I’m not making these numbers up, here are some articles I found about the change:

http://blog.conservation-us.org/blogpost.cfm?threadid=2227&catid=175
http://www.artdaily.org/index.asp?int_sec=2&int_new=38716
http://www.theartnewspaper.com/articles/Climate-control-time-to-change-the-settings/20913
http://www.theartnewspaper.com/articles/Revising-the-gold-standard-of-environmental-control%20/20549

And if any of my information is way off base, I certainly hope that my newest reader, Dixie, will set me straight.


Millage Campaign: Annmarie, Executive Vice President

I mentioned before that the DIA plans to launch a millage campaign in an effort to establish a more stable source of funding for the museum and that the initial poll results for the campaign were overwhelmingly positive.

Today, Annmarie presented a more detailed account of the poll results, which I found very interesting. I was asked not to share the details of the results, but I will discuss one thing that struck me about the poll and that was how many people reported a very positive image of the DIA. In one question, people were asked “Who goes to the DIA” and the common response was “Everyone”. I found this to be so encouraging. In a previous post, I said myself that I recognize museums (art museums, in particular) as having to struggle with that public perception of elitism. I think this just speaks to how many things the DIA is doing RIGHT. There are a lot of words that describe the City of Detroit (resilient, soulful, diverse, historic…) but I don’t think “elite” is one of them. Which is why I was so struck by the apparent perception of the DIA as being the people’s museum—a place where everyone goes. I think that this is a perception that most (all?) museums strive for and I am just so happy to see the community embrace the DIA as their own, just as I have for so many years.

But then again, I claimed ownership of Lake St. Clair when I was in preschool so I’m not sure I’m the best barometer of the community’s investment.  I get attached easily.

Granted

Today I met with a guy who had a $100,000 check in his pocket.  He is the DIA's new grant writer.

We met to discuss what kind of program information grant writers find useful when applying for grants.  As it turns out, the kind of information that would actually be useful is information that I will be unable to provide in my (now) 6 weeks at the DIA.  Awesome.

While it was somewhat frustrating to learn that my project will be less than helpful in the grant application process, it was still a very interesting meeting with David.  He gave me a great outline of the kind of information needed when writing grants.  Even if I am unable to gather that information during my internship, I think it is still valuable to know about.

One of the biggest things we talked about was "impact".  David cited this as being the most important thing to potential funders.  It is also the most difficult thing for museums to measure.  The kind of impact he is referring to are the long-term results of the program.

David described three kinds of results: short term, mid term and long term. 

Short term results involve things like customer service, visitor experience, and meaningful interpretations and tours.  The short term result of those things is a positive museum experience, or the visitor just having a positive feeling about the museum.
(Measurable with visitor surveys, etc.)

Mid term results spawn from short term results and involve things like visitors buying museum membership packages or any change in the visitors actions as a result of their feelings about the museum.
(Measurable by counting the # of memberships sold, etc.)

Long term results involve a social, environmental, or economic change in the community.
(Measurable... not at all.)

Grant writers want to be able to tell funders that "This program has improved children's critical thinking skills" or "This program has made the community more environmentally conscious".  But it is still difficult for museums to measure those kinds of things.  It seems that kind of long-term evaluation has yet to be perfected and implemented.

Either way, I really enjoyed meeting with David, as he had a lot of excellent insights.  However, while I did win Dixie's grant-writing contest last fall, and I would love to walk around with $100,000 checks in my pocket... I don't think grant writing is for me.

I Can't Believe I'm Enjoying Data Entry

I've been doing a lot of data entry the past few days.  But let me say this: data entry is not as mind-numbing when you're actually interested in the data itself.  I've been entering all of the programs from the past two fiscal years into categories for the AAMD mapping project.  My categories are things like schools, colleges/universities, religious, cultural, senior citizens, human services, libraries, businesses, corporate events, etc.

For me, it's been interesting to see where the majority of the groups are coming from.  So far, it's been a lot of Royal Oak, Farmington Hills, Bloomfield Hills, Dearborn and Detroit.  I expected more Grosse Pointe, but am happy to see the number of Detroit Public Schools that have visited the museum over the past two years.

Yesterday, I sat in on a department meeting, which was very interesting.  Apparently, the evaluation department is lumped into the human resources department at the DIA, forming something called "Organization Development and Human Resources" or ODHR.  That's where I work.  Now, because of this, I learned a lot about evaluation during our department meeting-- which I did not expect.

At the end of the semester, I worked on an evaluation project for the Harn Museum and I really enjoyed it.  Also, being in other meetings and hearing about visitor surveys has really piqued my interest in museum evaluation.

Apparently, the DIA has not always made evaluation a top priority, but in recent years, it has really taken off.  These days, every school group that comes into the museum gets a yellow envelope (reminiscent of end-of-the-semester evaluations) full of evaluations that both teachers and students complete before leaving the museum.  I didn't ask about it, but the amount of information collected from these surveys must be enormous.

Next week, I will be assisting with another type of evaluation-- an "employee engagement survey".  I'll be monitoring a few laptop computers in the Kresge court, where employees can take a survey about working at the DIA.  For some reason, it never occurred to me that satisfaction surveys could be internal.

I also learned that much of what the evaluation department does is dictated by grants.  And on that note, I'm happy to report that I'll be meeting with the DIA's brand new grant writer later this week.  He's a really nice guy and I'm excited to talk to him about writing grants.  I think I could find a niche for myself in grant writing, but we are meeting to discuss another project I'm working on (the creation of extended program summaries).  Sondra is concerned with making the program summaries into a resource that will be useful in grant applications and other campaigns.  Thus, I need to talk to the grant writer about what kind of program information would be useful in grants, etc.

On another note, I understand that the museum galleries and collections need to be kept and a stable temperature and RH, but is it really necessary for our office to be 40 degrees?  I am not a painting, and I don't enjoy sitting directly beneath an A/C vent.

But so far that is my only complaint about my internship at the DIA... so I guess things could be a lot worse.