Intranets Reimagined: Did we just statistically prove the case for a Social Intranet?

25 Aug Intranets Reimagined: Did we just statistically prove the case for a Social Intranet?

Co-Author Tyler Sauerteig

In partnership with the Worldwide Intranet Challenge, we’ve applied econometric principles to the results of over 200 intranet surveys and statistically pinpointed which intranet attributes most impact a user’s perception of their intranet. This third post in this seven week series is focused on the data analysis of  “intranet usage” factors — showing which intranet usage activities have the biggest impact on how employees view their company’s intranet.

Confirming the Basics

There are certain basic things employees have come to expect to be able to do on an intranet (e.g., reading company news). In this study, we dug a little deeper to see if we could identify which basic usage functions had the biggest positive impact on a employee’s valuation of their intranet.  With 99+% confidence, we can conclude that:

What Your Employees Want From Your IntranetFree Download: What Your Employees Want From Your Intranet

In this download, you will find the following information:

  • Ways to increase your employees’ rating of your company’s intranet
  • How to invest your intranet redesign money to get the biggest return
  • Key intranet fundamentals to ensure employee engagement
Get Your Free Download

The more often employees use their intranet to do these three things, the higher their overall valuation of their intranet:

      • Complete online forms
      • Upload or download documents
      • Find instructions for completing work tasks

Proving the Case for a Social Intranet?

For the last three years (if not more), software companies and social business advocates (self included) have been preaching the benefits and importance of making intranets “social”. In other words, providing users that ability to both post content and comment on others’ posts. Without getting into the pros in cons in this post, let’s just say that if we take in aggregate both social advocates and the more traditionally minded (and risk concerned) old guard … there is a “house divided”.  Again, we set out to dig deeper using the power of econometrics and see if we could show the impact of social activity on an employee’s valuation of their company’s intranet. With 94% confidence,  we can conclude that:

The more often employees use their intranet to provide feedback or comments about intranet content, the higher their overall valuation of their intranet.

These findings are based on Worldwide Intranet Challenge survey participants rating the importance of the following statements about the “their usage” of their company’s intranet.

      • Complete common work tasks such as applying for leave
      • Complete online forms
      • Publish content
      • Upload or download documents
      • Discuss work topics (eg. using discussion forums or blogs)
      • Collaborate with other staff (eg. using online work spaces)
      • Find instructions for completing work tasks
      • Provide feedback or comments about intranet content

Very Few Studies Are Perfect

We are making some big statements with this study, and while we are confident in the numbers, it is important that these statements are taken in the appropriate context. This particular analysis is showing that the more often employees use their intranet to provide feedback or comments about intranet content, the higher their overall valuation of their intranet. And we are taking that to mean that this “social” feature is positively impacting a employee’s valuation of their company’s intranet. But what about the other “social” features listed, namely, “discussing work topics” and “collaborating with other staff” … why didn’t these activities score higher? Well, one reason could be that the Worldwide Intranet Challenge started benchmarking intranets as early as 2009. How many intranets actually had the ability to discuss work topics and collaborate with staff five years ago? Another reason could be that companies generally tend to benchmark their intranets either prior to, or immediately after, a redesign. If the majority of companies benchmarked fall into the former category, it is unlikely that their intranets (at the time of the survey) had advanced social features. Both of these potential reasons lead me to believe that if our data set included more companies with intranets providing the ability to discuss work topics and collaborate with staff, these activities would have also scored high — providing further evidence that the more employees use the social features of their company’s intranet, the higher overall value they assign to it. Only time will tell. That said, it is our hope that by doing the best we could with the data available we have helped provide

“some solid quantitative evidence to support where companies should invest when completing an intranet redesign.”

We hope you are finding this study useful and look forward to your comments. We are fortunate to have had some top intranet experts weigh in on our evaluation of intranet “Look Factors” ... here’s to another great discussion.

To see the details on how we came to the above conclusions, please grab a (strong) cup of coffee and read the following section.

Behind the Numbers

For this category, we had to correct our equation for the heteroskedasticity within our model and we did so using White Heterosekdasticity-Consistent Standard Errors & Covariance. Heteroskedasticity causes the standard errors to be biased which leads to unreliable hypothesis testing. By using the consistent standard errors, we take a large amount of the bias out of the standard errors and our p-values are much stronger than they are with the regular estimation output. Heteroskedasticity does not bias the coefficients but the standard errors, so changing to standard errors helps our estimates greatly,and further limits our chance of error in our confidence levels. By using this corrected equation, all econometric problems are accounted for.

Statement                                                                                                                               Coefficient                 P-Value
Complete common work tasks such as applying for leave                   0.083                     0.230
Complete online forms                                                                            0.227                     0.002
Publish content                                                                                        0.047                    0.534
Upload or download documents                                                            0.322                     0.000
Discuss work topics (eg. using discussion forums or blogs                   0.010                     0.897
Collaborate with other staff (eg. using online work spaces)                 -0.077                    0.302
Find instructions for completing work tasks                                           0.207                    0.0025
Provide feedback or comments about intranet content                        0.149                     0.059

Adjusted- R²: 0.491

The adjusted-R² tells us that 49.1% of the variance in the valuation of the intranet in the WIC data is described by the statements within the category of “how often you use the intranet to”.

The statements that we found to be most important to your employees based on how often intranet usage to do certain things were: completing online forms, uploading or downloading documents, finding instructions for completing work tasks, and providing feedback or comments about intranet content.

      • With 99+% confidence, we can conclude that using the intranet more often to: complete online forms, upload or download documents, and find instructions for completing work tasks all increase the valuation ranking that your company’s intranet received.
      • With 94% confidence we can conclude that how often you use the intranet to provide feedback or comments about intranet content increases the valuation ranking that your company’s intranet received.

Interpretation of the Coefficient:

      • When looking only at the how often I use the intranet to statements, given your answer to the WIC statement “providing feedback or comments about intranet content” was ranked one rating higher (better) in the WIC ranking system for that statement, the way that you were ranked in the valuation question “In general, I would rate the intranet as” was on average 0.149 ranks higher (better), holding all other variables constant.

When looking at the data, even though they are not statistically significant, the variables dealing with collaboration are both negative. This is something to consider when building your intranet, but it is something beyond the intranet build that needs to be considered. Your employees still do not consider collaboration and discussion of work topics as important as those who adopt these internal networks and preach collaboration believe it should be. This is an issue greater than the intranet–it is something that needs to be continually preached within the company so that people are using the intranet to collaborate and become more efficient within the business. Your intranet needs to get to the point where your employees can really see the business value from collaboration on work topics.

Engagement based on intranet usage is something that you are really looking to provide to your employees. Therefore what makes your employees value your intranet highest, based on how often they are using it to do specific things, is vital to your intranet build and what kind of content you focus on. Our next blog post will be on the category of rating the intranet based on the effectiveness in providing specific aspects vital to your business and what aspects your employees find most important in their valuation.

Andy Jankowski

Andy is a Social Business researcher, advisor and keynote speaker. During the last 19 years he has served as a trusted advisor for several leading organizations including Andersen, Ernst & Young, JP Morgan Chase and Oracle. He is a career long student of enterprise communication and collaboration. He has both written for, and been written about, in Forbes and The Huffington Post. Andy is a frequent conference speaker and an avid road cyclist. He enjoys connecting people and dots.

More Posts - Website

Follow Me:
TwitterFacebookLinkedInGoogle Plus

9 Comments
  • Gabriele Sani
    Posted at 13:27h, 27 August Reply

    Very interesting results, but as far as I can see we lack information on what is the cause and what is the effect: the data could be equally seen as proof that engaged users are far more likely to provide feedback. If that is the case then the data tells us that one way to prove that users are engaged is to look at the volume of their feedback, which is still a very valuable result.

    Is there any evidence that enabling comments is the way to reach higher engagement? I would expect any modern intranet to allow comments, hence I do not expect that having the capability is enough… but I would not be entirely surprised if NOT having the ability to provide feedback would have a negative effect on user engagement.

    More info would be extremely welcome!

    • Andy Jankowski
      Posted at 13:39h, 28 August Reply

      Gabriele, Excellent points! I agree that in order to pinpoint the cause and effect of commenting (as well as impact) more info and analysis is needed. For example, could we correlate “before and after” employee engagement survey results in order to substantiate these and additional observations? Do you have additional insights (or data we could use) from your great progress at Oxfam? Or maybe know of others who are interested in working with us to dig deeper? Again, you raise great points. Thank you for you comments. I welcome any further discussion, but also appreciate you have limited time.

      • Gabriele Sani
        Posted at 11:09h, 15 September Reply

        Andy,

        Apologies, I saw your reply only when I came back to read your excellent article again!

        Your work does provide clear validation of the statement that 80% of the value of an intranet comes from the top 5 tasks (unfortunately I cannot find who said that…) which is a VERY important result.
        To be able to better communicate their value, can you look at the statistical distribution of your data for each one of those 5 tasks and give a numerical estimate of the value of having each one of them? Eg: being able to complete online forms improve the number of satisfied users by 25% (+/- 5%) I appreciate that it would be a bit of “dumbing down” your report to a mere snippet of what it is, but it would be absolutely great to put this number in front of senior management to prove the business case for developing those critical functions!!!
        By the way, do you have any data about people directory? I would expect it to be in the list of the must haves, too.

        Unfortunately I do not have much data on how we managed to get results: we work with continuous improvements, with releases that can be as short as daily, hence I cannot really see the individual impact of each project. Out of the top of my head, the only projects that had a visible impact on user behavior were:
        1) an internal marketing campaign where we used people profile pictures to create a mosaic with the name of our intranet, together with a “treasure hunt” with branded mugs as a prize for the winners.
        2) a change in our homepage: we removed an image carousel with links to three areas in our system with 3 static links and the frequency of search action with keywords related to those 3 areas dropped significantly (I was not smart enough to get a numerical value for this drop. I would say more than 50%, but it is a project from more than 3 years ago…) Besides, this proves only that the way our carousel was made was bad.

        Happy to provide mode details, or to discuss how we could use our system to dig deeper – you can get in touch via my email (not posting here to avoid spamming… but I hope you can see it associated with this comment. If not, you can get in touch via my linkedin profile at https://www.linkedin.com/in/gabrielesani )

        Thank you again!!!

        • Andy Jankowski
          Posted at 13:54h, 18 September Reply

          Gabriele, I followed up with our econometrics specialist. With regard to your question, “can you look at the statistical distribution of your data for each one of those 5 tasks and give a numerical estimate of the value of having each one of them?” The answer is yes, but unfortunately, this may not provide that much value because by omitting all of the other variable (to focus on one) we would be artificially creating bias. (This is a know constraint with econometric studies, reducing variables biases numbers). We can give it a try, and see what we can determine, but I fear it is going to be a very biased result.

  • Regan Sonnabend
    Posted at 20:18h, 27 August Reply

    Great post, Andy! Completing forms online, uploading/downloading documents, and finding instructions seems so basic–it is hard to believe it is really that easy to impact intranet valuation considering all the chatter these days. Unfortunately, my Dr. Pepper was not sufficiently caffeinated to allow me to channel my rusty Business Calculus skills and make sense of the stats section, but I’ll take your word for it. Thanks for the post!

    • Andy Jankowski
      Posted at 14:14h, 28 August Reply

      Regan, Thanks for the kind words! And yes, with all the chatter today (no pun intended), it is easy to overlook the basics. Also excited to see many of this industry’s new developments come to fruition — and hoping we can statistically prove their worth — we are fueled by espresso here at Enterprise Strategies 😉

  • Stephen Byrne
    Posted at 11:45h, 28 August Reply

    Hi Andy, Some of the effects you are describing are quite strong and I was wondering if you have any visuals of the relationships to make them more interpretable? Maybe some scatter plots?

    • Andy Jankowski
      Posted at 14:22h, 28 August Reply

      Stephen, Thanks for the kind words. At the end of this series we will be making available an “infographic” summarizing all of the findings. That, however, does not fully address your request. We are interested in, and have been working on, a good way of visualizing the econometric data/results we are compiling. We welcome any input, guidance (or help) regarding available tools and process — do you have experience in this area? As for now though, it is work in progress…

  • Proving the case for a task-based intranet | Intranet Diary
    Posted at 16:18h, 31 August Reply

    […] Enterprise Strategies post uses statistics from the Worldwide Intranet Challenge to analyse the aspects of an intranet […]

Post A Comment