March 25, 2013

Linux Sed Multiple Lines

Using sed to delete multiple lines from a file in linux

I remember how frustrating it was before I figured out the easy multiline sed command. This example is especially useful to clean up website hacks / injections - a "regular" single-line sed is fine to remove most of the common base64 injections that hit most of the files in the document root... But sometimes you come across JavaScript injections that are clearly "tagged" so there's an easy way to identify it in all injected files (even if there are 100,000+), but substituting part of the "signature" into your usual sed command just doesn't work.

Enter the multiline sed

There are two ways to do this, I'm going to go over the simpler one here, since if you are reading this, you probably need this information now:

Replace or remove multiple lines from a text file (the easy way):

sed '/FIRSTLINETODELETE/,/LASTLINETODELETE/d' /the/file/to/delete/from

*Lines can be regex, but make sure you get the whole line

Note: if you are cleaning script injections with a unique "tag" in the comment before and after the injection, just replace UNIQUETAG below with the random characters in the comment:

user@host [~]# sed '/<!--UNIQUETAG-->/,/<!--UNIQUETAG-->/d' /your/injected/file

This will output what the file will look like, but sed without -i will not modify the file.

If the output of the above looks correct, add -i after sed:

sed -i WILL modify the original file - please be sure your output is correct

sed -i '/<!--UNIQUETAG-->/,/<!--UNIQUETAG-->/d' /your/injected/file

"Regular" sed command, operates on a single line by default:

user@host [~]# sed 's/<!--UNIQUETAG.*UNIQUETAG-->//' /your/injected/file

This version will NOT work across multiple lines

Here's a test example that simulates an actual tag you may see in a javascript injection:

linux sed multi-line example command

*Note: I'm working in Centos but this should all work the same on most Linux distros. Also, for completeness' sake the other and "proper" way to do multi-line sed is the N option, which allows you to do much more complex tasks. This example deletes the last 2 lines of a file:

user@host [~]# sed 'N;$!P;$!D;$d' fileName

More multiline sed examples

In over your head? Call a Developer for help with Linux, servers, code, websites, and more. Let us worry about the tech stuff while you move your business forward.


"Linux Sed Multiple Lines" continued here...