Coding: Bring the Noise

%26lt;0011 Em

A binary serialization mixin for Python: More Metaclasses (Part 4)

leave a comment »

In our last episode, we had a concise field wrapper that let us defer object instantiation so we could call the initializer (with all the args and kwargs specified at class declaration) each time we instantiated the class we were building.  We still didn’t have anything related to serialization.  Today we finalize the metaclass and take stock of our final solution.  Spoilers: there’s a better solution than Metaclasses, but the investigation’s worth it.

Like the previous post, we’ll be focusing on the third point:

  1. Many values fit in one or two bytes at most – booleans fit in a bit, color channels fit in 1 byte. Any solution needs to allow bit-width specification.
  2. However, I shouldn’t need to specify the bit-width of every value. Reasonable defaults should exist that can be dropped in where appropriate.
  3. Serialization shouldn’t get in the way, and I shouldn’t have to give it much thought when I write my classes.
  4. Of course, I should be able to drop into the details when a class requires custom serialization.
  5. Performance overhead, both time and space, should be as low as possible. Per-instance overhead is almost entirely unacceptable.
  6. Serializable classes should be nestable, and the performance overheads for nested classes should be as low as possible.

Read the rest of this entry »

Written by delwinna

May 28, 2013 at 4:45 am

PyProMan: virtualenv wrapper for multi-project development

leave a comment »

Developing with SublimeText continues to be a pleasure, except that curious interaction with virtualenv and ST’s build system.  I seem to have two choices:

  1. Create a new build system for each virtualenv – which is pretty much 1:1 for my projects.  When switching between projects in ST, don’t forget to change the build system, too.
  2. Use 1 build system for python, but every time I switch projects, edit the build system with the proper python target.  No initial setup, but I’ve got to keep the build system open and remember where the venv is.

Enter PyProMan.  Basic, but it gets the job done and there’s almost no overhead if you decide later that you don’t want to use it anymore.

tl;dr

Put your projects in the $PROJ folder, and get started with a new virtualenv by running “proj pythonX.X” from that folder.  For the project “foo” there will be a new virtualenv in $VENV/foo.  Now, you can interact with any projects’ appropriate bin/ scripts by putting “proj” before the command (much like sudo).  Want to install a package for the project you’re in?  “proj pip install some_package” and it’s done.

But wait there’s more

The real reason this was put together was to make the SublimeText integration trivial.  Since our build system will always run the script we’re building from the project folder it lives in, we can use the same exact build cmd for all projects.  We don’t have to create and manage multiple build systems, we don’t have to edit it when switching projects.  “proj python $file” will figure out the appropriate python interpreter and hand it the rest of our arguments.  And we can use that command outside of the project folder as well, and still use the right virtualenv.  As long as you have put proj before python when executing a script, you can be sure it’s executing in the right environment.

Wow!  Anything else I should know?

Not really – two environment variables and two small bash scripts (that could be one).  For more information including my full SublimeText build system, check out the readme at https://github.com/numberoverzero/pyproman

Written by delwinna

May 26, 2013 at 8:03 am

Posted in Uncategorized

A binary serialization mixin for Python: Metaclasses (Part 3)

leave a comment »

Like part 2, this post will focus on points three and five:

  1. Many values fit in one or two bytes at most – booleans fit in a bit, color channels fit in 1 byte. Any solution needs to allow bit-width specification.
  2. However, I shouldn’t need to specify the bit-width of every value. Reasonable defaults should exist that can be dropped in where appropriate.
  3. Serialization shouldn’t get in the way, and I shouldn’t have to give it much thought when I write my classes.
  4. Of course, I should be able to drop into the details when a class requires custom serialization.
  5. Performance overhead, both time and space, should be as low as possible. Per-instance overhead is almost entirely unacceptable.
  6. Serializable classes should be nestable, and the performance overheads for nested classes should be as low as possible.

This post we’re only going to create a metaclass that leverages some of the magic of the Field class – next post we tackle serialization-specific work in the metaclass.

Read the rest of this entry »

Written by delwinna

May 19, 2013 at 8:48 pm

A binary serialization mixin for Python: Descriptor protocol (Part 2)

leave a comment »

This post we’re going to focus on the third point of our requirements. To recap, here they are:

  1. Many values fit in one or two bytes at most – booleans fit in a bit, color channels fit in 1 byte. Any solution needs to allow bit-width specification.
  2. However, I shouldn’t need to specify the bit-width of every value. Reasonable defaults should exist that can be dropped in where appropriate.
  3. Serialization shouldn’t get in the way, and I shouldn’t have to give it much thought when I write my classes.
  4. Of course, I should be able to drop into the details when a class requires custom serialization.
  5. Performance overhead, both time and space, should be as low as possible. Per-instance overhead is almost entirely unacceptable.
  6. Serializable classes should be nestable, and the performance overheads for nested classes should be as low as possible.

We’ll first take a quick look at the code the last post finished with – hopefully the problems we found can help design an interface that more closely fits the third point, while keeping the fifth (performance) in mind.

Read the rest of this entry »

Written by delwinna

May 19, 2013 at 8:12 am

Posted in Descriptors, Python

A binary serialization mixin for Python: Requirements (Part 1)

leave a comment »

I’ve been working on a project that requires serializing objects.  I could use Google’s protobuf, but this seemed like a great opportunity to see what I could do with python.  Of course, less painful serialization isn’t a new concept.

I have some preferences for how this serialization fits in with my existing code, and a few of them have been notably complex:

  1. Many values fit in one or two bytes at most – booleans fit in a bit, color channels fit in 1 byte.  Any solution needs to allow bit-width specification.
  2. However, I shouldn’t need to specify the bit-width of every value.  Reasonable defaults should exist that can be dropped in where appropriate.
  3. Serialization shouldn’t get in the way, and I shouldn’t have to give it much thought when I write my classes.
  4. Of course, I should be able to drop into the details when a class requires custom serialization.
  5. Performance overhead, both time and space, should be as low as possible.  Per-instance overhead is almost entirely unacceptable.
  6. Serializable classes should be nestable, and the performance overheads for nested classes should be as low as possible.

Read the rest of this entry »

Written by delwinna

May 18, 2013 at 10:32 am

More thoughts on the Collatz Conjecture

leave a comment »

I made a quick housekeeping sweep of the blog recently, and wanted to revisit the Collatz Conjecture.  I’m sure it’s been suggested before, but I believe there’s another method of proving the conjecture:

Collatz(k) for all k \in [2^{n}, 2^{n+1}] where n \in \mathbb{Z}^+ > 2

That is to say, we only need prove that for an n > 2, all integers between consecutive powers of 2 resolve according to the collatz conjecture.  I’m not sure where to go from there, but for instance we can show that in general:

2^n + 1 \Rightarrow 3^{r} + 1 where r is the number of (divisible by 3) iterations such that n = 2r

Read the rest of this entry »

Written by delwinna

May 13, 2013 at 2:15 am

Posted in Puzzles

C#/XNA: Heatmaps

leave a comment »

I ported the Iterative Diamond-Square Algorithm from Python, and added a few features.

random_generation

The biggest addition is interactivity, and the biggest optimization is using shaders for updating/drawing.  If I have time in the next few days, I might try to move the random generation to a shader.  Shouldn’t be to hard, and it would be a significant gain in speed.  In the meantime, I’ve got generation running on a separate thread (thank you C#’s anonymous functions!)

The code is available on Github.  Follow the break for usage instructions, screenshots, and code highlights.

Read the rest of this entry »

Written by delwinna

May 7, 2012 at 10:04 pm

Posted in Projects

Follow

Get every new post delivered to your Inbox.