All posts by David Veksler

Multithreaded queue process with C# & BackgroundWorker

And now for something completely different:

This weekend, my mail server was slammed by a spammer using a rogue account to create hundreds of thousands of spam emails that jammed my outbound mail queue. Mixed with the spam were valuable customer emails, so I had to sort through all the mail ASAP and delete anything that wasn’t legit.

First I tried a simple loop that loaded each file and deleted it if it contained a bad string. But that was taking a while, so I made my filter multithreaded.

First, I load a list of files to process:

string[] files = Directory.GetFiles(directory);
Console.WriteLine(files.Length + " files.");

(You can iterate through the files instead, but I wanted to see how many files there are.)

I instantiate the class with the BackgroundWorker:

DeleteProcess DeleteProcess = new DeleteProcess();

Now, I loop through the files, checking each for spam:

            foreach (string mFile in files)
            {
                if (CheckBlacklist(mFile))
                {
                    DeleteProcess.filesToDelete.Add(mFile);
                    if (!DeleteProcess.worker.IsBusy)
                        DeleteProcess.worker.RunWorkerAsync();
                }
            }

Instead of loading the whole file, I just read it until I determine that it is spam. Since 99% of messages were spam, this went pretty quickly:

 private static bool CheckBlacklist(string mFile)
        {
            using (StreamReader reader = new StreamReader(new FileStream(mFile, FileMode.Open, FileAccess.Read)))
            {
                string line;
                while ((line = reader.ReadLine()) != null)
                {
                    if (line.Contains("NIGERIA") || line.Contains("Message Delivery Delay"))
                        return true;
                }
            }
            return false;
        }

(By using FileAccess.Read, I speed things up a bit.)

Now for the delete thread. Here is how it’s wired up:

public List filesToDelete = new List();
 
        public BackgroundWorker worker = new BackgroundWorker { WorkerReportsProgress = true, WorkerSupportsCancellation = true };
 
        public DeleteProcess()
        {
            worker.DoWork += worker_DoWork;
            worker.ProgressChanged += worker_ProgressChanged;
            worker.RunWorkerCompleted += worker_RunWorkerCompleted;
        }

The worker thread should get the first file name from the queue, delete the file, and then delete the filename list item:

 private void worker_DoWork(object sender, DoWorkEventArgs e)
        {
            while (filesToDelete.Count > 0)
            {
                worker.ReportProgress(0, filesToDelete[0].Replace(Program.directory, string.Empty));
                File.Delete(filesToDelete[0]);
                File.Delete(filesToDelete[0].Replace(@"OutgoingMessages", @"Outgoing"));
                filesToDelete.RemoveAt(0);
            }
        }

When we’re done, we count the remaining files:

Console.WriteLine(Directory.GetFiles(Program.directory).Length + " files left.");

It’s possible to create a collection of BackgroundWorkers if you want to utilize multiple CPU’s, but the bottleneck in this case was the disk IO, so it wouldn’t help.

AES interoperability between .Net and iPhone

One of my projects requires encrypting data on the iPhone and decrypting it using .Net. This is easy to do with the Common Crypto library in the iPhone SDK and the AesCryptoServiceProvider class in .Net, but the encryption parameters have to be the same for it to work.

I couldn’t figure it out, but the geniuses at StackOverflow did, so I am posting my results here. The zip file includes a basic iPhone app and a .Net console project with helpful classes to do the encryption/decryption and base64 conversion. I didn’t write most of the code – thanks to Blue Beetle for the .Net code and Greg Haygood for the Objective C.

Download zip.

Why no Paint?

macpaintWhy is there no basic paint application in OS X? Sure, there are dozens of free applications that do the same thing, but it seems like a useful thing to have for users who don’t need anything more complex.  After all, a drawing application was the first application created for the Macintosh, before even the operating system.  I’m sure there is a philosophical or marketing reason behind it, but I can’t think of any.

Most adult Windows users who use Microsoft’s Paint actually use it to save screen shots, which is easier in OS X with Preview’s “Grab” menu or keyboard shortcuts.

Running you first .Net app on OS X

Installing Mono will allow you to run .Net applications in OS X – as long as they are 100% .Net and do not use any native Windows API’s. However, if you try to simply double-click on a .Net exe, OS X will not know what to do with it, or if you have VMware or wine installed, try to open that executable in another application.

To open a .Net exe with mono, you must open a terminal window, switch to the application directory, and type “mono Application.exe.” If you get an error stating something like “System.DllNotFoundException: gdiplus.dll” it’s probably because your application uses System.Windows.Forms, which must run under X11.  To test X11, install the latest version and run the app again from the X11 terminal.  Did it work? Great!

You probably don’t want to open X11 and type a command in terminal every time you want to run a .Net application, so you can make a script to do it for you.  Open ScriptEditor (/Applications/Utilities/AppleScript/) and create a new script.  Type something like:

do shell script "mono /Users/YOU/Downloads/YourApplication.exe"

Save the script as an Application and you’re done!  Now you can run it just like any native OS X app.  You can try it with SharpChess – just download the exe version.  I was able to download the source of SharpChess, compile it with MonoDevelop on OS X and ran the exe I made on both Mono/OS X and Microsoft .Net/Windows.  Unfortunately, Mono’s implementation of WinForms does not use the native Cocoa API, so it doesn’t look very good – I’ll work on that later.

Customizing Terminal when compiling Mono apps

Pkg-config is a helper tool used when compiling applications and libraries.  If you want to build Mono apps from source using configuration scripts, you will need to put the Mono.pc path in your PKG_CONFIG_PATH environment variable.  If it’s not set, you will get an error like “configure: error: missing the mono.pc file, usually found in the mono-devel package” or “Failed to initialize the ‘Mono 3.5 Profile’ (mono-3.5) target framework”

Here’s how to customize your terminal prompt.   To add the location of mono.pc, edit .profile or .bashrc in the root of your home folder, and add this line

export PKG_CONFIG_PATH=/Library/Frameworks/Mono.framework/Versions/Current/lib/pkgconfig/"

Here is my full .profile file:

export PATH=/Applications/Windows/Darwine/Wine.bundle/Contents/bin:/opt/local/bin:/opt/local/sbin:$PATH
export PKG_CONFIG_PATH=/Library/Frameworks/Mono.framework/Versions/Current/lib/pkgconfig/
export DISPLAY=:0.0

(When using MacPorts, the path is /opt/local/lib/pkgconfig/)