Take the 2-minute tour ×

I have to perform a large change to a codebase that consists of a handful of different types of changes that need to be applied in hundreds of different locations, spread out over hundreds of thousands of lines.

I have an idea for a tool that can help me with this, but as I'm sure I'm not the only one with this idea, I wonder if it's already been written.

Let me outline how it would work:

  • First, a bit like grep with context, it would collect a set of "interesting" chunks of code based on a regular expression; there may be thousands of these locations.
  • Then, let me iterate through chunks, marking each in turn as Interesting or Uninteresting. This is basically automating as much as possible of the manual job of whittling down potential change locations to actual change locations.
  • Finally, let me apply a transformation (e.g. sed-style substitution) to all selected Interesting locations.

Does this tool already exist?

I'm thinking of writing this tool myself if I can't find a pre-existing one.

share|improve this question
If you can't find a suitable tool, and have to build one yourself. It'd be wonderful if you open sourced it. There have been plenty of where I would have loved to have a similar tool to what you're proposing. –  sdolan Nov 18 '10 at 17:56
add comment

4 Answers

up vote 1 down vote accepted

This sounds similar to what Coccinelle was written to do, although it only works with C.

share|improve this answer
That's interesting, but I think Coccinelle's approach is probably too structured to be practical; in other words, it tries to automate too much, but the complexity pops up on the other side in writing the semantic patches. Because the change set doesn't need to be reusable, I think it's better to make pragmatic one-off manual choices where it makes sense. –  Barry Kelly Nov 18 '10 at 18:53
add comment

Sounds interesting. I've also played around with such idea but didn't went far from command-line scripting. My approach for refactoring preparation is:

  1. Find a chunc of code to be refactored
  2. Generate reg-exp/script for finding similar code/pattern and create a list of location for this type of pattern

The output-file contains lines in GNU or MS output format (e.g: FILE:LINE MESSAGE) So it can be loaded in any IDE (of vim -q) and the code-chunks can be found easily by double-klicking on "Error messages".

By the way, it is easier to grep, if the code was unified by indent previously.

share|improve this answer
add comment

I'm not aware of any tools like that. It seems like a fairly specialized task that's only done once in a great while, so it's hard to make money by developing and distributing such a tool.

In the past, if I had a task like that, I would write a script in Emacs' version of Lisp. The advantage there is that Lisp is a powerful language, and the Emacs editor has many convenient built-in functions (e.g. query-replace-regular-expression) and concepts. However, unless you're already familiar with Emacs and Lisp, I wouldn't recommend it; the learning curve is too steep.

share|improve this answer
The tool I have in mind would be a 200-line hack-job to minimize total work time of (tool-creation + refactoring), rather than some polished product. query-replace-regular-expression isn't much more than sed's s command; as a first pass, my tool will likely be extracting chunks to be modified into a large text file, whittled down by hand, then modified with sed, awk etc., then reintegrated with the originals. –  Barry Kelly Nov 19 '10 at 1:37
add comment

It sounds like you plan to do this by using hueristics ("grep") for finding your code, and heuristics ("sed") for modifying your code. If these will do the trick, and you can really do it in 200 lines as you say, I'm surprised you even asked here on SO.

As a general rule, making hundreds of changes using heuristics is a rather dangerous. If a person is involved in each one of them, he can fix the errors to the extent he notices them and that may be good enough; in that case you are building just a funny looking text editor. If you go that route, EMACS might be a very good choice, as all the actions you want (string search, extract-to-buffer-for-display, string replace, build marker data structures on the side) are completely scriptable in Elisp and it already has a nice UI.

If you want to automate this on a more reliable basis, you need accurate searching and replacement. Our DMS Software Reengineering Toolkit is a program transformation tool that can do this for many widely used languages (you didn't say which one you were doing) including Java, C++, C, C#, COBOL, ... DMS can likewise be fully scripted for custom sets of actions.

share|improve this answer
Well, I can do it in less than 200 lines, but it gets progressively harder to write succinctly, and involves more manual steps :) - reviewing the output of grep -n -C 10 redirected to a text file, then post-processed into a diff, and finally applied with patch. But a visual interface designed around this kind of workflow - and it is a workflow problem by nature - would make life much easier. Automating it isn't the point; the whole issue with it is the repetitiveness of the examination of each individual case. –  Barry Kelly Nov 25 '10 at 4:13
add comment

Your Answer


