Friday, July 2, 2010

How to parse multipart/form-data request on an ASPX backend from a non-ASPX client

Implementing File upload through an .NET ASPX client is fairly straightforward. On your
ASPX upload page, you add your file upload markup like this..





.. to post the uploaded file to the server side where processing the upload request is equally as simple: To get the uploaded file, Type System.Web.HttpPostedFile does the trick and similarly to get any other passed values, HttpContext.Request.Params or HttpContext.Request.QueryString or HttpContext.Request.Form depending on how you passed them or how specific you want to get about retrieving.

But, how would your process an upload from a non-ASPX clients?. I will
describe how I achieved it from a Flash client where the upload request posts from the Flash object instead of an ASPX page. After some digging, I came to the understanding that my multipart/form-data payload from the Flash client is going to be in the form of some kind of ASCII text/binary combo and I would be responsible for parsing out the image
and parameter values..in other words, no convenient utilities to my aid.
Essentially, I have to craft my own HttpRequest data parser as HttpPostFile
or Request.Param are no good in this case.

So I rolled up my sleeves and after
scavenging bits and pieces from online posts and picking the brains of colleagues, this is how I put this puzzle together. A rough breakdown of the steps are:

1) Get the multipart data as a byte array:
byte[] multiPartDataBytes=HttpContent.Request.BinaryRead(HttpCotext.Request.TotalBytes);

2) Convert the byte array to a string as It is easier to break apart the byte
array using indexes calculated from string. Proper string construction is
important as described below.

3) Parse the multipart data byte array for parameter name / value pairs and the image.
Parsing out the image binary is the interesting part of this exercise so Iwill focus mainly on that.

Now, you need some understanding of the layout of your multipart data to parse it and the simplest method of gauging this is to dump the request to a text file and just look at it. File.WriteAllBytes(path, multiPartDataBytes) works nicely as it converts the ASCII bytes to string.

I noticed that my binary image data was sandwiched between two ASCII text strings: "Content-Type: application/octet-stream\r\n\r\n" and a boundary string. The boundary string is easy to spot as well. It looks something like this: "------------KM7ae0Ij5GI3gL6gL6cH2Ef1Ef1Ij5" it's purpose should be obvious.

One gotcha I discovered here is that when I first tried to get the boundary string from HttpContent.Request.Headers.ToString() by parsing for string "boundary=----..blahblah", I noticed that the boundary string multiPartDataBytes(defined above) contains a few extra hyphens. I suspect this is normal but seems contradictory to me and I didn't bother investigating why the header's version of the boundary was different from the actually boundary value used in the body of the request data. I decided to derive the boundary string from the body (multiPartDataBytes) instead of the header as it is the body that I would be parsing anyway.

So back to the image parsing: It is a lot easier to look for my "octet-stream" and boundary markers in a string instead of a binary array so I next converted my multiPartData byte array to a string..but be careful..your want to preserve the ordinal string positions so that the indexes match the indexes in the byte array.

To convert to the string:

System.Text.Encoding myEncoding = System.TextEncoding(1252):
string multiPartDataString = myEncoding.GetString(multiPartDataBytes, 0, multiPartDataBytes.Length);


I don't fully look into why encoding 1252 works best but it ensures that a value at an index in the string array matches up to the same index position in the byte array and also that no two bytes are converted to the same string value...very important. Now I just look for the index position of my "Content-Type: application/octet-stream\r\n\r\n" value (start index) and then the index position of my boundary (end index). You want to account for any line feeds and carriage returns as you don't want to pick those up when the pull out the image binary.

In C#, here is how I found the start index:
string imgStartStr = "Content-Type: application/octet-stream" + Environment.NewLine + Environment.NewLine;
long startIdx = multiPartDataString .IndexOf(imgStartStr , StringComparison.Ordinal);
Notice the StringComparison.Ordinal option? This goes to making sure the index positions are accurate. EndIdx is calculated in a similar manner.
Once I have the start and end indexes, I take that to my multiPartDataBytes array and extract those bytes between those positions. I found that Array.Copy(..) worked great for that. Once I had the bytes for the image I used File.WriteAllBytes(...) to complete the upload to the server.

Parsing for any other parameter name / value pairs is just as easy once you know the layout of the request body. In my case there was no other binary data involved and the name values came in the request at the top of the body while the octet-stream was always at the bottom..not sure if it will stay like this always but you want to make sure that your parser is flexible enough to deal with different layouts of the request data.