ORM’s Dreaming Big: Pt 3 (Big Pappa ORM)

Here is Part 1 and Part 2 if you are interested

Two Orms to date have been very interesting to me.

  • RethinkDB – Pushes events to any listeners. This inherently supports cluster since if a socket attempting to synchronize, it needs to know the changes. Luckilly, the changes will be pushed to all threads.
  • Waterline – What I really like about waterline is that instead of being a storage system, it is just the interface to it. This allows you to have specific storages and maximize their best parts without having to write in different query languages.

Databases have gone in and out of style so fast over the past years. MongoDB, CouchDB, Postgres, Hadoop, MySQL. All of which are competing for the marketshare of being the “database of choice”. That being said, All of them have distinct advantages and disadvantages. Anything SQL also gives you the ability to run a PHP stack without much trouble. Anything JSON allows you to store documents in the format your coding in. Additionally, Redis has shown that moving the global memory of sessions and the like to a single store is very important for clusters. As a result, the Storages increase and unfortunately, the interface increases as well.

Queries, Adapters, The Bazaar and the Cathedral

If you havent read this essay, I think you should. The gist of it is there is a clear difference in the way people write parts and something is created in one huge gulp at a time. Now, I’m in the parts boat because as time continues, there may be new Databases. There may be new fancy things. And as a result, you don’t want your api to change with each database you move to. In a bazaar, you have your fruit farmers, butchers, jewlery, etc. Each is specialized with a solid respect to each person and what they do. In a cathedral, you are given your daily bread and wine on special occassions. This is simple and works however, some cathedral’s bread is better than others. And sometimes you get olive oil with yours. I believe that allowing as many databases to interact with your ORM is superior to forcing a the API to adhear to your Databases design. In this way, you can make the Query Language the best Possible without breaking compatibility with an Databases. As a result, there is a few simple laws I believe he ORM should adhear to

  • One Query Syntax for all databases it interfaces with
  • “Adapters” decompile the query into what the database will actually use and send it

As result, there becomes a dependency. Adapters -> Models. However, Adapters are more abstract, they generally should be reused for multiple databases of the same type. As a result there becomes a further dependency.

Adapters -> Connection to a Database -> Model/Table

Pluggin in, Plugging out and letting everything just Figure itself out

Content management systems are allowing you to design databases in the browser. WordPress has custom PostTypes. Drupal has Schemas. Treeline is making “machines”. However, the most important concept here is that when you make an edit to the a database model, the whole server doesn’t have to shut down on the way there. PHP has the advantage of not existing in memory as a result, each call to the server is essentially a “fresh” instance. NodeJS doesn’t have that kindness. As a result, making sure your orm is designed in such a way that dependencies don’t require the need to Destroy and recreate is of upmost importantce. Something simple such as

orm.addModel("ModelName", modelConfig)
orm.removeModel("ModelName");

Can really make or break what an ORMs capable of. A simplistic Algorithm would be something like this.

util.inherits(ORM,EventEmitter);

ORM.prototype.addModel = function(name, config){
  this.models[name] = new Model(config);
  var connDeps = getConnectionDeps(config);
  var modelDeps = getModelDeps(config);
  allListeners(name,modelDeps,this,"model");
  anyListeners(name,connDeps,this,"connection");
}



function allListeners(name,deps,orm,key){
  var depnum = 0;
  var addlistener = function(){
    depnum++;
    if(depnum === 0){
      orm.emit(
        "add-"+key+"["+name+"]",
        orm[key][name]
      )
    }
  }
  var remlistener = function(){
    if(depnum === 0){
      orm.emit(
        "rem-"+key+"["+name+"]",
        orm[key][name]
      )
    }
    depnum--;
  }

  deps.forEach(function(depname){
    if(!orm[key][depname]){
      depnums--;
      orm.on(
        "add-"+key+"["+depname+"]",
        addListener
      )
    }else{
      orm.on(
        "rem-"+key+"["+depname+"]",
        remListener
      )
    }
  });
  orm.on("destroy-"+key+"["+name+"]",function(){
    deps.forEach(function(depname){
      orm.off(
        "add-"+key+"["+depname+"]",
        addListener
      )
      orm.off(
        "rem-"+key+"["+depname+"]",
        remListener
      )
  });
}

function anyListeners(name,deps,orm,key){
  var depnum = 0;
  var addlistener = function(){
    if(depnum === 0){
      orm.emit(
        "add-"+key+"["+name+"]",
        orm[key][name]
      )
    }
    depnum++;
  }
  var remlistener = function(){
    depnum--;
    if(depnum === 0){
      orm.emit(
        "rem-"+key+"["+name+"]",
        orm[key][name]
      )
    }
  }

  deps.forEach(function(depname){
    if(!orm[key][depname]){
      orm.on(
        "add-"+key+"["+depname+"]",
        addListener
      )
    }else{
      depnums++;
      orm.on(
        "rem-"+key+"["+depname+"]",
        remListener
      )
    }
  });
  orm.on("destroy-"+key+"["+name+"]",function(){
    deps.forEach(function(depname){
      orm.off(
        "add-"+key+"["+depname+"]",
        addListener
      )
      orm.off(
        "rem-"+key+"["+depname+"]",
        remListener
      )
  });
}

With even emitters, you can toggle back and forth with minimal issues. Theres other parts that are important such as…

  • Binding a model to the orm instead of making the call itself
  • Being able to queue requests from a model
  • Throwing errors when things fail

Cluster Support

Cluster support is one of the most important parts about any modern javascript module now adays. If it can’t be run on a cluster, its not fit for production. If its not fit for production, its going to end up being just fgor somebodies side project. From a simple concept, you can add cluster support by relaying events. This simple example is all we really need to ensure we are sending events properly. First off we must figure out what events need to be sent globally. For our cases, we’ll do delete of a model

ORM.prototype.addModel = function(){
  var model = this.figureOutDeps(arguments);
  var self = this;
  model.on("delete", function(instances){
    if(self.isChild){
      process.send({
        type:"orm-rebroadcast",
        event:"model["+model.name+"-delete"
        data:instances
      });
    }
    self.emit(
      "model["+model.name+"-delete",
      instances
    );
  });
}

As you can see, when a delete has happened locally, we then tell the master what has happened. From here the master tells every other worker what has happened

ORM.prototype.asMaster = function(workers){
  var self = this;
  workers.forEach(function(worker){
    worker.on("message", function(msg){
      if(msg.type === "orm-rebroadcast"){
        self.broadcast(msg,worker);
      }
    });
  });
  this.workers = workers;
}

ORM.prototype.broadcast = function(msg,not){
  this.workers.forEach(function(worker){
    if(worker === not) return;
    worker.send(msg);
  });
}

From there we can implement the worker’s listeners

ORM.prototype.asWorker = function(){
  var self = this;
  process.on("message", function(msg){
    if(msg.type === "orm-rebroadcast"){
      self.emit(
        msg.event,
        msge.data
      );
    }
  })
}

There are things that can be done a little bit nicer. For example, having workers tell the master what the want to listen and not listen for. Additionally, We can reimplement this with Redis or any other api because it really isn’t that complicated.

MVC as applied to wordpess and a moment of silence for XSL

So, finishing up the backbones of the Clockin Plugin, I’ve found the next steps are to see it as I had it on my old site. With a multitude of charts and such. However, the display for the clockin plugin isn’t static. In fact its quite dynamic. The entire idea of a “clock in” is its date as such it will be changing almost daily. This is ok and unfortunate for me in that it was what bogged down my load times before. I gotta do what I got to do.

Now there are a few things I’m going to be doing

  1. Add a specific ui thats just for clock_in_project
  2. Add a specific ui thats just for users, and allow them to be searchable
  3. Start Moving away from the 2014 base (while attempting to keep a lot of the great parts)
  4. Create my old charts
  5. Cache days that are not current days for easy access

But my to do this isn’t why I’m writing this blog post, the reason I’m writing this is to explain what a model view controller is and what wordpress offers.

The best way I can explain the model view controller is this:

  • Model is an object in your database
  • View is how you’ll be viewing the Model
  • Controller is directing what Model to choose and what view to choose for it
    -This can be based off: User Request, Admin Request, App Request and Default

With wordpress, our MVC is simple: we have post types, we have a theme, we have a url.
Whenever a user makes a request at the top thereurl, what we’re actually doing is making a request.
Now from my experience there are two main forms of requests, archive, singles. Archives will show you a list of items based off a query. Singles may also be based off a query, the difference is that you get full frontal information instead of a bunch of items.

Why does this matter?

Well for the clock in post type, we’re going to have to do some very special things. And Unfortunately as is the default it renders as if its just a regular post. We’re going to change that.

Its something very simple, where we can base it off of file structure. But I’d prefer not

switch (get_post_type( get_the_ID() )){
case "type_a": break;
case "type_b": break;
default: echo "ah, the good old default";
}

As simple as this we can go through and pick our view. we can also mix and match and a bunch of funky things.

One thing I want to point out is a sad sad truth about MVC, XSLT. XSLT was designed to make HTML the view component of choice. Where we would take in our model, see what we had and apply the appropriate templates. Important to note, it was all (well kinda) XML. It gave people the conceptual graduation platform to move up to. So why didn’t it work out?

XSLT has a major flaw, most Databases don’t return XML. If your database isn’t returning XML, your going to have to Transform it into XML. If you’re transforming, your wasting time. Especially when you can just use the raw data retrieved to fill in the dots. Another Major flow is that despite how awesome Xpath is, there isn’t a direct correlation from url concepts o XPath concepts.

What people need to understand is the url is the command, just as XPath is a command. And most people go from HTML, CSS, File system into PHP and Javascript where either they sink or swim. XSL and Xpath had the opportunity to make that bridge easier. But they died as other CMS’s popped up and took over. Will we see them again? One day maybe, I certainly try to use it when i can. But I highly highly doubt it

A ReEntry to WordPress

I build wordpress for clients mostly, but had big plans for my own website. However, after a month of work and a lot of research I’ve come to understand what it really means to… create a content management system. There are many things to keep in mind, access to your database, providing a nice looking interface. Making things simple yet also allowing complete customization. The User Object. Which should be static methods, which should be instance? There is a lot too it, and I finally had decided that getting my website up and running is far more important than spending years on a website that (no offense to websites) is not very high on my priority list.

What I will say though is
1) CMS’ are meant to be easy to install, easy to begin and easy to use
-Wordpress is absolutely adorable compared to what I had been working on so hard for most of my workday.
2) CMS’s data are meant to be transportable
-Now wordpress isn’t necessarilly transportable, however you can definitely export your data as a solid XML file and due to its popularity importable as well
-for my own work, its definitely obvious I won’t be accessing the data quite so soon. Exporting to XML is the easy part, importing is a completely seperate beast
3) If your going to make a user interface make you can use it again
-for me I wanted to use XSL badly, and I still do. But the complexity in automation turned an interesting idea into a beast of its own. I’ll talk about it later when I get back into it. However, the important parts isn’t the XSL but rather the HTML. css and javascript
4) Routing is one if not the most important part of a CMS
-User Permissions are based off what is asked for
-Automation is very very important when dealing with an unknown amount of objects with unknown methods
-These are two things that held me back the most

Now I can’t say I want to use wordpress, but I definitely prefer it currently over slaving away for hours inorder to give myself more hurdles. Having the requirements change, introducing new technology mid project, these are things I’ll have to deal regardless. However, to me, finishing something is far more important than making it perfect.