Click here to Skip to main content
13,351,239 members (47,697 online)
Click here to Skip to main content
Add your own
alternative version


4 bookmarked
Posted 11 Oct 2013

Research-Inspired Extensions Hit Visual Studio App Store

, 11 Oct 2013
Rate this:
Please Sign up or sign in to vote.
A peek into the inspiration behind two cutting edge Visual Studio extensions.


There are TONS of extensions in the Visual Studio App Store (i.e., Visual Studio Gallery), most of which are small improvements on VS's default tooling and, in my opinion, not worth the hassle of installing.  However, there's a new class of extensions, inspired by years of software engineering research, that show real potential to enhance developers' performance. Today I'll present one commercial extension and one free open source extension, both inspired by research, and I'll discuss how developers can use these tools to dramatically improve their performance on common development tasks.

Continuous Testing Research   


While making changes to a mature code base it's easy to accidentally break existing features, causing a regression. In fact, it is so common that while at MIT in 2004 David Saff created an (Eclipse) plugin to automatically run tests in the background and let developers know as soon as they'd broken an existing test. The driving principle being that the sooner any regression is caught the less of a productivity drain it will be, echoing the concept that the earlier in the lifecycle a bug is found the easier it is to fix. Not surprisingly, when they tested this idea in a user study developers performed much better when supported by a continuous testing tool. As you can see above, almost three times as many developers completed the task when given continuous testing tool support (i.e., 78% completed with tool support, only 27% without), relatively strong evidence of its usefulness.

Continuous Testing in Practice 

Fast forward to 2012 and not much has changed in testing. While it was more likely that your team had a continuous integration server, which may notify you after checking in a regression, Saff's original vision of instant notification has largely gone unrealized. Fortunately, Remco Mulder and team have since released NCrunch for Visual Studio. This extension, as shown above, provides decorators on every line of executable code (green, red, or black dots) that indicate (a) whether that line is covered by at least one test and (b) whether all tests executing that line are passing. It continuously runs tests in the background each time there is an update to code, and thus developers are aware of regressions almost immediately after they happen. Based on Saff's original research developers should see a measurable gain in performance thanks to this early warning system. 

Software Search Research

Anyone who has programmed in Visual Studio has used FindInFiles (above) at one time or another. Like its predecessor 'grep', FindInFiles searches each file in a solution and matches the user query against each line using regular expressions. Unfortunately, that means it is slow to execute and good search queries are extremely hard to construct. So hard, in fact, that in a 2006 study of developers performing maintenance tasks 88% of user's queries failed to find anything relevant. Fortunately, since about 2004 many researchers have been working on this issue. Andrian Marcus was among the first to use information retrieval techniques (think Google and Lucene) to index and search source code, making it much easier to search local code bases. Adding to this progress, researchers (e.g., Emily Hill and Sonia Haiduc) have investigated how to help users create better queries, one of the biggest challenges in code search.

 Software Search in Practice 

Again, fast forward to 2012 and not much had changed in software search. Developers were still struggling to use FindInFiles, scrolling through screens of irrelevant results. That's when a university professor, an open source contributor, and I came together to create the free, open source Sando Code Search Tool for Visual Studio.  Sando uses Lucene.NET to index source code and thus can search code almost instantly, providing both a keyword based search (e.g., [add child] as shown above) as well as literal search via quotes (e.g., ["//add person as a parent"]). It uses some of the latest research to recommend relevant search terms as users type. Our data shows that developers get significantly better results and are happier with those results when they use these recommendations. Based on previously mentioned studies, where developers' searches failed 88% of the time, using an advanced search tool like Sando can dramatically improve developers' productivity, especially when working on a new or unfamiliar code base.

Note: Another great Lucene-based search tool for Visual Studio is Entrian Source Search.

 Which Extensions to Use? 

As a former extension junkie, who downloaded any Visual Studio Gallery extension that was even vaguely relevant, I'll never stop experimenting with new extensions. However, for my stable set of core extensions I now have a higher standard. I look for extensions that will save time or increase code quality. If I can't make a clear argument as to why I have an extension installed when asked by a colleague, it doesn't make the cut. While I may be a bit extreme, I'd encourage you to put a bit more thought into which extensions you install. Don't believe every snake oil salesman hawking their extension; look for those who built extensions based on lessons from the past.  

Downloads: NCrunch, Sando


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

David C Shepherd
Technical Lead
United States United States
David C. Shepherd is a software engineering researcher and research area coordinator with ABB, Inc. based in Raleigh, N.C. While working with ABB's Software Engineering Group he looks into improving developer productivity and increasing software quality. While he is a researcher, and thus often found writing papers and attending research conferences, his industrial background compels him to actually implement his ideas as industrial tools.

You may also be interested in...


Comments and Discussions

BugNice idea and works for smaller solutions... Pin
Member 956675515-Oct-13 6:28
memberMember 956675515-Oct-13 6:28 
GeneralRe: Nice idea and works for smaller solutions... Pin
David C Shepherd21-Oct-13 4:29
memberDavid C Shepherd21-Oct-13 4:29 
GeneralRe: Nice idea and works for smaller solutions... Pin
John B Oliver6-Nov-13 11:25
memberJohn B Oliver6-Nov-13 11:25 
GeneralRe: Nice idea and works for smaller solutions... Pin
David C Shepherd18-Dec-13 10:18
memberDavid C Shepherd18-Dec-13 10:18 
GeneralRe: Nice idea and works for smaller solutions... Pin
David C Shepherd1-Feb-14 6:20
memberDavid C Shepherd1-Feb-14 6:20 
GeneralRe: Nice idea and works for smaller solutions... Pin
David C Shepherd9-Sep-14 5:53
memberDavid C Shepherd9-Sep-14 5:53 
QuestionHas been approved Pin
Kenneth Haugland11-Oct-13 9:53
memberKenneth Haugland11-Oct-13 9:53 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.180111.1 | Last Updated 11 Oct 2013
Article Copyright 2013 by David C Shepherd
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid