Category Archives: Measuring Culture

Word Clouds of Organisational Communication: Poor Man’s Sentiment Analysis?

wordmap

This is just an idea I have not yet had the chance to implement so, sadly, cannot share any results.

The idea is this: if we collect all the contents from communications in an organisation, say from instant messaging, and present the results in a word cloud does this provide a useful insight? I’m aware that for more sophisticated sentiment analysis context is very important but inside an organisation the context is more-or-less fixed, both from the perspective of those writing the communications and those observing the resulting word cloud. Could it work, does anyone have any experience of try this or similar analysis?

 

When a social network knows it is being watched does it change?

cctv-camera

There is tremendous value in analysing social networks, both internally to an organisation, and looking at the social networks of the organisation’s customers, suppliers, industry influencers etc. but what happens when that community becomes aware that their publications and interactions are being analysed?

I asked this question at Social Data Week ’13 in London  and the panel’s answer was that it did: you can already observe people taking advantage of this in expecting some sort of reward for following, or otherwise being associated with, organisations. This sounds fairly innocuous but I am more concerned about observing networks inside the organisation.

Think about the following scenario: an organisation analyses IMs to gauge sentiment and presents this information by department; one department noticeably has a lot of negative sentiment compared to the others; the department’s manager is advised of this and asked to devise and implement a plan to improve the situation. What could they decide to do, the three options are:

1         The right thing: find the root causes and address them

2         The lazy thing: don’t do anything, hope it improves

3         The wrong thing: tell members of the department the communications are being monitored and not to use negative language

I have actually observed the wrong thing being done when it comes to staff surveys (which amongst other things are trying to gauge staff sentiment about the organisation): the manager of the department let it be known they did not want to see negative ratings of management in the survey, presumably because the results of the survey had some bearing on their bonus. I fear the same thing would happen if social network analysis and/or sentiment analysis were being used.

Another option an organisation has is to use surveys to build a picture of the social network (I’ve recently exchanged some views with TECI who take this approach). In this case its clear that the organisation is collecting the data but I wonder how accurate this is; I think people may either not answer entirely honestly or simply forget about certain connections in their network as they don’t seem important (but could be very important in the overall network). I’d love to know if anyone has any studies that compare networks derived from surveys with those derived from communications data. My guess is doing both and combining the results would give the most accuracy.

So if an organisation does want to use communications data for SNA what should it do? Having thought about this I think the answer if to firstly baseline the communications data and then announce that the organisation has such an intent (assuring staff that it will be analysed anonymously) and finally observe the communications data to see if there is a change from the baseline. The next step depends on the result: if there is very little change then it’s probably OK to carry on but if there is a noticeable change then this is telling the organisation something and it needs to understand why there was a change before proceeding.

Does anyone know of any studies, or have any experience of, social networks changing if they become aware they are observed?

 

Is Instant Messaging being used to pass notes in your meetings?

boredI have observed a couple of behaviours around Instant Messaging (IM) in meetings: the first is IM being used to contact people not in the meeting, either to ask for information pertinent to the meeting, are being asked a question or just to chat about something unrelated because they are bored; the second behaviour is to use IM internally in the meeting where one sub-group will be making comments whilst another is talking. But which of these is more typical and why is this useful to know?

To understand what I’m talking about take a look at this example:

meeting

Here we represent four people in a meeting (inside the ellipse) and two people outside the meeting. Lines between the people indicate that they have shared one or more IMs during the meeting. The first chart, below, shows the average number of people involved in IM conversations  for meeting sizes between 2 and 17 people; the sample is about 2000 meetings over 3 months; there are very few meetings with more than 17 people so these are excluded:

IMs in meetings average number of people

Not unexpectedly as meetings get larger there are a few more people using IM in meetings. The ratio of internal to external IMs looks fairly constant though, but this is best observed with another chart:

IMs in meetings average number of people ratio

The ratio is indeed fairly constant at nearly 50:50.

I’ve previously looked for correlations between meeting sizes and number of instant messages (as opposed to the number of people using IM) and not found much. Taking the same approach with the number of people, which is to divide the number of people using IM by the number of people in the meeting we get the following:

IMs in meetings average number of people per attendee

Which, interestingly, shows IMs dropping off on a per-person basis, as the meetings get larger. Is this an indicator of cultural behaviour or is it simply that IM often gets used to tell people the conference call isn’t working / lost the PIN / the phone is broken / can’t use the computer etc.? To answer this would require some analysis of message content, which is possible, but not in the scope of this study. [Complete aside: talking about conference calls check this out]

So why bother? Well most organisations have a bunch of cultural objectives along the lines of “let’s be nice to each other” and, if some further analysis of IMs inside meetings shows that they do not reflect this aspiration, then you can measure your actual culture and observe the trend to see if your cultural change initiatives are working or not.