Intimidating MONKEY

Random bites from a code monkey

Setting Up Unit Testing with AppEngine

Apr 19

Posted by Monkey in Python | No Comments

So, being completely unconvinced of my own ability to produce bug free code, the first thing that came up when working with AppEngine was: how do I unit test my app?

I found a post in one of the Google groups (linked at the bottom) that discussed it but was either somewhat wrong or just out of date.

So the following worked for me:

The following was done with the standard Python 2.5 install for OS X, along with the AppEngine SDK for OS X.

It turns out that in addition to the imports I was using for my code I needed:

from google.appengine.api import apiproxy_stub_map
from google.appengine.api import datastore_file_stub
from google.appengine.api import mail_stub
from google.appengine.api import urlfetch_stub
from google.appengine.api import user_service_stub

I also had to adjust the python path so I could get my unit tests to run outside of the dev_appserver instance.

export PYTHONPATH=/usr/local/google_appengine/:/usr/local/google_appengine/lib/django/:/usr/local/google_appengine/lib/webob/:/usr/local/google_appengine/lib/yaml/lib/

Keep in mind that the export above would be needed to be explicitly added to any startup scripts you’re using in order for it to be persistent.

Then the, hopefully, obvious import:

import unittest

Then there’s the step of setting up the environment for each test to run my setUp method follows:

def setUp(self):
# Ensure we're in UTC.
os.environ['TZ'] = 'UTC'
time.tzset()
# Start with a fresh api proxy.
apiproxy_stub_map.apiproxy = apiproxy_stub_map.APIProxyStubMap()
# Use a fresh stub datastore.
stub = datastore_file_stub.DatastoreFileStub(APP_ID, '/dev/null', '/dev/null')
apiproxy_stub_map.apiproxy.RegisterStub('datastore_v3', stub)
# Use a fresh stub UserService.
apiproxy_stub_map.apiproxy.RegisterStub('user', user_service_stub.UserServiceStub())
os.environ['AUTH_DOMAIN'] = AUTH_DOMAIN
os.environ['USER_EMAIL'] = LOGGED_IN_USER
# Use a fresh urlfetch stub.
apiproxy_stub_map.apiproxy.RegisterStub('urlfetch', urlfetch_stub.URLFetchServiceStub())
# Use a fresh mail stub.
apiproxy_stub_map.apiproxy.RegisterStub('mail', mail_stub.MailServiceStub())
self.current_user = users.get_current_user()

The setup method was borrowed from this groups post

Want to know how long a block of code took to run?

Jul 25

Posted by Monkey in Ruby, Tech | 1 Comment 

def timeThis
    startTime = Time.now()
    yield
    endTime = Time.now()
    endTime.to_f() - startTime.to_f()
end
puts(timeThis{sleep(0.25)})

Displays (standard out)0.25

Find a DOM child element by name (Javascript)

Apr 20

Posted by Monkey in Tech | 1 Comment

Please, someone tell me there is a built in way to do this. But in the interim, since I don’t know of another way, if you’re looking for a child of a given node (by name), you can do it something like this.

    // Used to store the matched node, if found.
    var _jsMatchedNode;
 
    function jsFindChildByName(pvNodeName, pvParent)
    {
        if (pvParent.nodeName.toLowerCase() == pvNodeName.toLowerCase())
        {
            _jsMatchedNode = pvParent;
            return;
        }
 
        if (!pvParent.childNodes) return;
 
        if (pvParent.childNodes.length == 0) return;
        var nodeList = pvParent.childNodes;
        for(var x=0;x<nodeList.length;x++)
        {
               jsFindChildByName(pvNodeName, nodeList[x]);
        }
    }

Of course if you use this repeatedly it would behoove you to reset the _jsMatchedNode variable before each use.

Thou shalt not explicitly DataBind

Apr 19

Posted by Monkey in ASP.NET, C# | No Comments

Mmmmm. Remember that the Page.DataBind does this for you, explicitly calling control.DataBind is probably a bad idea.

See:
Do you use Page.DataBind?
Avoid calling Page.DataBind and bind each control individually to optimize your data binding. Calling Page.DataBind recursively calls DataBind on each control on the page.

You’re best bet is to override DataBind and call base.DataBind()

C# Array Initialization

Apr 7

Posted by Monkey in C# | No Comments

An important reminder from our sponsors (oh, and just admit it… You can never remember this anyway):

From the 1.2 language spec:

The new operator automatically initializes the elements of an array to their default value, which, for example, is zero for all numeric types and null for all reference types.

Important notes about Array.Initialize()

It must not be used on reference-type arrays.
…
You can use this method only on value types that have constructors; however, value types that are native to C# do not have constructors.

And of course, if you don’t trust me, see for yourself.

List<> Interleaving

Apr 5

Posted by Monkey in C# | No Comments

Ever wanted to interleave multiple sources of data? Well I did, so I wrote this, which interleaves multiple (generic) lists into a single.

public static List<telement> Interleave<telement>(params List<telement>[] pvParamLists)
{
    // if nothing is provided we'll simply return an empty list
    if (pvParamLists == null) return new List<telement>();
 
    int listCount = pvParamLists.Length;
 
    // offset map indicating current position in a given list
    int[] pointerMap = new int[listCount];
 
    // count total elements
    int resultSize = 0;
    foreach(List<telement> list in pvParamLists)
    {
        if (list != null)
        {
            resultSize += list.Count;
        }
    }
 
    // create our result list
    List<telement> interleavedList = new List<telement>(resultSize);
 
    // will quit when finished
    while (interleavedList.Count < resultSize)
    {
        for (int x = 0; x < listCount; x++)
        {
            if (pointerMap[x] != -1)
            {
                List<telement> currentList = pvParamLists[x];
                // skip null or empty lists
                if ((currentList == null) || (currentList.Count == 0) || (pointerMap[x] >= currentList.Count))
                {
                    pointerMap[x] = -1;
                    continue;
                }
 
                interleavedList.Add(currentList[pointerMap[x]++]);
            }
        }
    }
 
    return interleavedList;
}

The whole file, including some unit tests is here.

Copy via XmlSerializer

Mar 27

Posted by Monkey in .Net, C#, Tech | No Comments

For God’s sake, don’t ever write this (at least not again). Its such a trivial piece of code that I never bother librarifying it, and yet every time I write it I leave out the reset of the stream position. But, what do you expect… I’M A MONKEY!!!

public static Object XmlSerializationCopy(Object pvValue)
{
  if (pvValue == null) return null;
  XmlSerializer ser = new   XmlSerializer(pvValue.GetType());
  // most are probably small.
  MemoryStream serStream = new MemoryStream(512);
  using(serStream)
  {
    ser.Serialize(serStream, pvValue);
    serStream.Position = 0;
    return ser.Deserialize(serStream);
  }
}

File.GetLastWriteTime vs. File on Network (ding, ding, ding)

Mar 27

Posted by Monkey in Tech | No Comments

So, I was wondering why file size (FileInfo.Length) seemed to be more accurate indication of a file modification than File.GetLastWriteTime. I was aware of the fact that the time updated on closing of the stream doing the writing to the file in question, and was willing to accept that. The question was where the heck was this called out…

Here is where it is called out.

Specifically:


When writing to a file, the last write time is not fully updated until all handles used for writing have been closed. Therefore, to ensure an accurate last write time, close the file handle immediately after writing to the file.

The part that is extra sucky is that this update of the Last Write Time does seem to take place on local files (not accessed via a remote UNC path), that is the Last Write Time does seem to change on local files even while the stream changing it is still open.

Use StreamWriter with XmlSerializer

Mar 27

Posted by Monkey in .Net | 1 Comment

Or something similar lest you risk paying the price for not paying attention.

The XmlSerializer can serialize an object to a stream, any stream. If the stream you are searializing your object to already contains data, and the length of that data is longer than the length of the data you are currently writing (your about-to-be serialized object) you get artifacts.

This is to say that if you open a stream without clearing it in some manner, and then write less data than what was already there…. You get your new data at the begining, and leftovers from what was there before at the end.

So in the case of serializing to a file, the easiest thing to do is use a StreamWriter and provide the path of the file to which you wish to write. The StreamWriter overwrites any existing file for you by default.

As per msdn documentation:
StreamWriter Constructor (String)
The path parameter can be a file name, including a file on a Universal Naming Convention (UNC) share. If the file exists, it is overwritten; otherwise, a new file is created.

Log4Net (Log4x?) File Locking

Mar 26

Posted by Monkey in Tech | No Comments

While I have yet to delve deeply into it, the default locking of log files by log4net hinders concurrent reads. The ExclusiveLock locking model (default) seems to keep the change determination based on last write, or file size to fail (or at least not work in a timely manner). I know there is a whole bunch of detail around this that I haven’t looked into, however setting the locking model to MinimalLock addressed the problem as I was seeing it (allowed concurrent reads to detect changes).