Kohana Forums

Hmm, interesting. Always looking for alternative DB systems for some problem cases in my site :) Do you have any use case examples where you're using Mongo? Or in what situations you might use Mongo over the traditional MySQL/PostgreSQL etc?

Will have to do some more research on Mongo then :) Mainly I'm looking for something to handle large forums and full text searches, though

As I'm getting more and more understanding of MongoDB and the document-based way of thinking, I made many improvements to my Mango library.

Now - besides the RDBMS like features (mostly all relationships that ORM also supports), I added some more fancy features:

To push a value into an array, call $object->set[] = 'some value'; - even if $object->set has not been set yet.

To increase a counter, call $object->counter->increment() - even if $object->counter has not been set yet

I also added support for a multidimensional array of counters. You could do: $object->site = array(
'views' => 0,
'clicks' => 0,
'pages' => array(
'12' => array(
'views' => 0,
'clicks' => 0
)
)
);

Save it,

increment/decrement when needed:
$object->site['pages']['12']['views']->increment(5); (this corresponds to a $inc => { site.pages.12.views : 5 } update)

and show it when needed:
echo 'this site has' . $object->site['total'] . ' views';

Nice :-)

I know, don't you like my very original name?! :-)

I ported my libraries to KO3. I figured that anyone trying MangoDB (which is in release 0.97) probably also uses KO3 :-). The KO3 version will be the one that gets all the updates.

I would like it if one of the admins could open a Mango project for me on projects.kohanaphp.org so I can list it over there as well.

Anyway, see http://github.com/Wouterrr/MangoDB and http://github.com/Wouterrr/MangoDemo

Do you have any examples? Where do I put the files? I am new to Kohana.

Thanks,
Bill

@Wouter

I am testing out your Mango modules (thanks for the hard work), and I will like to know if your ACL and A2 modules also work with Mango.

Thanks in advance.

Nano.

@Wouter

I have been playing with your demo and I really like the flexibility you get from Mongo and the nice features you have added to Mango (relationships, cascade deleting, etc...), just wondering: do I need to set all the columns in advance in my models? All the examples I have seen have the different columns already defined in the models, but somewhere in the MongoDB page I saw that Mongo is quite flexible for adding new columns on the fly. Any help will be appreciated.

Thanks,

Nano.

Is there any reason why the save method of Mongo is not implemented in Mango? I see that there are update, insert, delete, but I don't see the save method.

Thanks,

Nano.

@rafi,

but create does an insert [1] internally; save [2] in the other hand will try to update the object if in the collection, otherwise will do the insert.

[1] http://us2.php.net/manual/en/mongocollection.insert.php
[2] http://us2.php.net/manual/en/mongocollection.save.php

Thanks Wouter, I've seen lately that ActiveRecord-like modules (eg. new ORM, Sprig) have the 2 methods insert/update instead of a single save one.

By any chance have you migrated your A1 module to extend from Mango instead of ORM?

Nano.

I think it will take a little bit more than just changing that info, but I will definitely give it a try, I was hoping to save some time :). After long time looking for my best options I have decided to use MangoDB instead of MySql and your modules are very helpful.

Just out of curiosity, have you used Mango in any real-life web project?

Thanks,

Nano.

@nano, I'm using Mango/MongoDB in my current project, it's awesome :) There's some big names using Mongo as you can see from Wouter's link.

As far as the A1 with Mango, I took a look in Wouter's repository, and it looks very simple to implement. Just remove the comments :) I didn't get to the login/authentication yet, but I don't see any problems.

@nano: I'm using Mongo as my primary database

Great you're using the module! And thanks for including your code. It makes reproducing much easier.

In fact, all the things your experiencing come down to a single issue. Arrays and sets, so I'll try to explain that one first.

PHP is kind of unique by not only supporting numerical arrays array('value0','value1'), but also associative arrays array('key' => 'value'). All array related PHP methods run on both types of arrays.

In other languages, an array is always numerical. Otherwise you're dealing with an object. This also goes for MongoDB. MongoDB uses a JSON like storage mechanism. Documents are stored like this

{
_id : ObjectId('1234'),
some_array: [ 'value0', 'value1'],
some_object: {
key : 'value'
}
}

I made the following translation from MongoDB arrays and objects to PHP's numerical arrays and associative arrays:

MongoDB array = PHP numerical array = Mango_Set
MongoDB object = PHP associative array = Mango_Array

In your Model_Test, the some_array field is actually a set! Therefore, the correct specification would be:

<?php
class Model_Test extends Mango {

protected $_fields = array(
'some_set' => array('type'=>'set', 'type_hint'=>'array'),
'name' => array('type'=>'string')
);

protected $_db = 'demo'; //don't use default db config
}

All your troubles come down to the differences between arrays and sets ( = MongoDB objects and MongoDB arrays) and how MongoDB deals with them. So fix that, and try to redo your tests :-).

Some notes:

1) Updating sets using dotnotated array indeces doesn't work properly in Mongo. Bug is reported here: http://jira.mongodb.org/browse/SERVER-181
2) I could not recreate this error. If should work properly once you change your 'array' to 'set'.
3) Same issue array/set thing. If you specify a field as a Mango_Array, but then add only numerical keys, the MongoDB driver (not Mango, but the driver itself) will assume the document is an array because it only sees numerical keys, and stores it as such.

@Wouter: Hey. Good explanation.


1) I'm trying to understand when to use or not use the type_hint. Couldn't quite fully understand from the wiki.
2) Arrays/Sets:

-------+------------+----------------------
Mango | Mongo | PHP
-------+------------+----------------------
Set | Array | Numerical array
Array | Object | Associative array
-------+------------+----------------------


Pretty confusing. Did I get it right? :-)

BTW,
@nano - make sure you read the 32-bit limitation of MongoDB.

Dude. That was like the best answer in the world. I just smoked a little and you blew my mind :)
We could store whole object inside, and also have tons of attributes as sets, inside a single array field, and have those attributes act as sets and in each one store a set with i18n locales for multi super editable multi-translated website without dealing with all the freakin' SQL for medium websites like blogs or stores, so cool man! :^)

I was thinking if MongoDB is basically JSON, how can I get a JSON string directly from the load() method ?
I'm using a few javascript picker widgets that I want to feed with json over xmlhttp.

Thanks a zillion!!!

I want to share the following.

I have this model

class Model_Person extends Mango {

protected $_fields = array(
'siblings' => array('type'=>'set', 'type_hint'=>'array')
);

}


Creating an entry is fine:
$person = Mango::factory('person');
$person->siblings[] = array('name' => 'Mary', 'last_name' => 'Smith');
$person->create();


But updating will not work, instead it ALWAYS creates a new object for the siblings set.
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0] = array('name' => 'Jane', 'last_name' => 'Smith');
$person->update();


Same here:
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[] = array('name' => 'Mary', 'last_name' => 'Smith');
$person->create();
$person->siblings[0] = array('name' => 'Jane', 'last_name' => 'Smith'); //Even if values are EXACT size as original values
$person->update();


> db.persons.find()
{ "_id" : ObjectId(MongoId), "some_set" : [
{
"name" : "Mary",
"last_name" : "Smith"
},
{
"name" : "Jane",
"last_name" : "Smith"
}
] }


Changing the model to use array instead of set, it will work (it updates the value instead of adding entries to siblings object, but with the condition that values are the exact number of characters):
class Model_Person extends Mango {

protected $_fields = array(
'siblings' => array('type'=>'array', 'type_hint'=>'array')
);

}


This will work:
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0] = array('name' => 'Jane', 'last_name' => 'Smith');
$person->update();


This will not work: (not same length for attribute name)
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0] = array('name' => 'Mary Jane', 'last_name' => 'Smith');
$person->update();


This doesn't update anything either: (missing 'last_name')
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0] = array('name' => 'Jane');
$person->update();


However, if I update the value directly by itself it will work (it works for sets as well)
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0]['name'] = 'Jane';
$person->update();


The main limitation with this approach is that values cannot grow or shrink in size, thus, updating ONLY works when the same number of characters are updated.

Notice that adding new key/value pairs doesn't work either. This will fail (I think this is the case for the bug issue that Wouter mentioned):
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings[0]['middle_name'] = 'Jane';
$person->update();


However, there is a way to modify the values of the keys. You MUST use array type and set a key name for each entry.

This will work:
$person = Mango::factory('person', array('_id' => MongoId))->load();
$person->siblings['eldest'] = array('name' => 'Mary', 'last_name' => 'Smith');
$person->create();
$person->siblings['eldest'] = array('name' => 'Mary Jane', 'last_name' => 'Smith');
$person->update();
$person->siblings['eldest']['name'] = 'Maria';
$person->update();
$person->siblings['eldest']['middle_name'] = 'Jane'; // Adding new key is possible
$person->update();
// Be careful when copying objects
$person->siblings['youngest'] = $person->siblings['eldest']; // Copying existing object
$person->siblings['youngest']['name'] = 'Joe'; // This updates the key 'name' for $person->siblings['eldest'] as well
$person->siblings['eldest']['name'] = 'Maryann'; // This updates the key 'name' for $person->siblings['youngest'] as well
$person->update();


My conclusion is that if you want to have an object that you think you are going to edit, then use arrays with a non-numerical keyname, but if you are collecting values (probably a log) then using sets is fine. When copying objects around be careful if you are updating the values, you can get some surprises there!

Thanks,

Nano.

@Wouter

Thank you for your post, always a good learning source.

Is this a typo? (I haven't dig too much into extensions yet)

http://github.com/Wouterrr/MangoDB/blob/master/classes/mango.php#L514

Shouldn't it be?
protected function _set_model_definition(array $definition)
{
if(isset($definition['_fields']))
{
$this->_fields = array_merge($this->_fields,$definition['_fields']);
}

if(isset($definition['_relations']))
{
$this->_relations = array_merge($this->_relations,$definition['_relations']);
}
}


Or is _has_one the only relation accepted when using _set_model_definition?

Thanks,

Nano.

Hi Wouter,

I didn't see the count method for collections implemented on mango, thus, this is what I have done and it worked for me.

Method count on mango.php:
/**
* Return the count of a (set of) document(s) from the database
*
* @param array specify additional criteria
* @return int the result from count
*/
public function count(array $criteria = array())
{
// Not sure if should restrict it to embedded
if($this->_embedded)
{
throw new Mango_Exception(':name model is embedded and cannot be loaded from database',
array(':name' => $this->_model));
}

$criteria += $this->changed(FALSE);

if ( isset($criteria['_id']))
{
// if ID is set, we don't need any other value
$criteria = array(
'_id' => $criteria['_id']
);
}

// resets $this->_changed array
$this->clear();

return $this->_db->count($this->_collection,$criteria);
}


And mangodb.php also needed some adjustments because count expects only 1 parameter:
Replace this:
public function count( $collection_name, array $query = array(), array $fields = array() )
{
return $this->_call('count', array(
'collection_name' => $collection_name,
'query' => $query,
'fields' => $fields
));
}


With this one:
public function count( $collection_name, array $query = array())
{
return $this->_call('count', array(
'collection_name' => $collection_name,
'query' => $query
));
}


And also at the __call method:
case 'count':
$r = $c->count($query,$fields);


for this one:
case 'count':
$r = $c->count($query);


And its usage (assuming there is a log model with an field called date, which preferable is an index for faster responses):
$visits = Mango::factory('log', array('date' => '2010-01-01'))->count();


Sorry I have not created a patch, kind of lazy :)

Let me know your thoughts.

Thanks,

Nano.

I somehow agree that having a count at the ORM-like level might not make sense, but Mango is not a traditional ORM library, it deals with a different type of database. To me the count method for MongoDB should be considered because of the performance difference. If I don't have a method for getting the counts then I have to retrieve all the objects and after that do a $total = count($model_results) which consumes memory (even if limiting the fields to be retrieved). If we are talking about counting entries in the order of magnitude of thousand of them, then I think it could matter.

In addition, I feel count should be considered because of the nature of MongoDB, they provide that API method because the database favors that kind of queries. The count is not done via find, which in relational dbs the count is done via select statements. My 2 cents.

What do you consider could be an alternative to get the counts for many objects right now? Is there a way to do some kind of raw calls to db from Mango?

Thanks,

Nano.

@Nano: The MangoDB class already supports the count method. I'm only wondering if it should be included in the Mango ORM layer.

This will always work:
$db = MangoDB::instance(); // or $log->db();
$visits = $db->count('logs', array('date'=>'2010-01-01'));

I was eagerly awaiting the $unset and the $set on array indices features, so although MongoDB 1.3 is still a development version, I did add support to Mango for those features (+ I did some code cleanup). Please update your MongoDB installation to 1.3 to be able to use the new features.

There is some testing to do, as it can get pretty complex (eg with nested arrays and sets), and MongoDB does not support some combinations of modifiers on a single field. Also, there could be bugs in Mango and in MongoDB, so if you run into something strange, post it here, and we'll try to look for an answer :-).

@wouter: wow! that's bunch load of commits! you like that space after the parenthesis in if huh? :-)
Can you update mangodemo with some new examples? :P

@nano: I'd like to know about one too. Webfaction seems nice and you can install Mongo, but they won't support it.

I cannot remove an element from a set. I am trying this but doesn't work. ($model->my_set[0] already exists);
$model = Mango::factory('test')->load(1);
unset($model->my_set[0]);
$model->update();


Model:
class Model_Test extends Mango {
protected $_fields = array(
'my_set' => array(
'type'=>'set'
)
);
}


Any hints?

Thanks.