IRC logs of #tryton for Friday, 2011-03-25 #tryton log beginning Fri Mar 25 00:00:01 CET 2011
-!- elbenfreund1( has joined #tryton01:22
-!- FWiesing( has left #tryton01:46
-!- heg( has joined #tryton02:17
-!- gremly(~gremly@ has joined #tryton02:20
-!- serpent213( has joined #tryton03:57
-!- cheche(cheche@ has joined #tryton04:00
-!- XeBa( has joined #tryton04:00
-!- marga( has joined #tryton04:00
-!- heg( has joined #tryton04:45
-!- silverfox1971( has joined #tryton05:05
-!- heg( has joined #tryton05:10
-!- digitalsatori(~tony@ has joined #tryton05:10
-!- Vladimirek( has joined #tryton05:16
-!- yangoon( has joined #tryton05:18
-!- okko( has joined #tryton07:24
-!- enlightx( has joined #tryton07:52
-!- trifon( has joined #tryton07:57
-!- plantian( has joined #tryton08:07
-!- predatell(~werty@ has joined #tryton08:14
-!- udono( has joined #tryton08:32
-!- Timitos(~kp@ has joined #tryton09:06
-!- Mayank(~MnzNotebu@ has joined #tryton09:16
-!- ecarreras(~under@unaffiliated/ecarreras) has joined #tryton09:19
-!- nicoe( has joined #tryton09:20
-!- okko(~okko@ has joined #tryton09:20
-!- curlynostrill(~curlynost@ has joined #tryton09:25
-!- digitalsatori(~tony@ has joined #tryton09:31
-!- paepke( has joined #tryton09:42
-!- cedk(~ced@gentoo/developer/cedk) has joined #tryton09:53
-!- bechamel( has joined #tryton10:09
-!- trifon_( has joined #tryton10:22
-!- ikks_(~ikks@ has joined #tryton11:05
-!- tony_(~tony@ has joined #tryton11:32
cedkyangoon: is it you who has wrote a script to generate translation file for country?11:42
yangooncedk: yes11:46
yangoonisn't it anywhere in codereview or tracker?11:47
cedkyangoon: because there is this issue
cedkand I'm wondering if we could use it to generate those new translations11:48
yangooncedk: I don't know if subdivisions are in iso-codes11:48
cedkyangoon: they come from the module pycountry which uses the ISO11:49
yangooncedk: yes, the base is iso-country from debian11:50
-!- svaksha(~svaksha@unaffiliated/svaksha) has joined #tryton11:52
cedkyangoon: ok I think we should put it in country module under scripts11:53
cedkyangoon: and update it to translate also the subdivision11:53
-!- trifon_( has joined #tryton11:56
yangooncedk: phone12:06
yangoonyes, good idea, same for currency12:06
yangoonjust now short in time, bbl12:06
-!- pjstevns( has joined #tryton12:20
-!- enlightx( has joined #tryton12:32
-!- hoRn( has joined #tryton13:19
-!- silverfox1971( has left #tryton13:31
-!- lem0na(~lem0na@ has joined #tryton13:40
-!- tony_(~tony@ has joined #tryton13:53
-!- chrue( has joined #tryton15:21
-!- nicoe_( has joined #tryton15:38
-!- Vladimirek( has joined #tryton16:15
-!- nicoe( has joined #tryton16:37
-!- zodman(~andres-va@foresight/developer/zodman) has joined #tryton16:49
-!- Mayank(~MnzNotebu@ has joined #tryton16:59
-!- nicoe( has joined #tryton17:04
-!- paepke( has joined #tryton17:11
-!- elbenfreund( has joined #tryton17:32
-!- okko(~okko@ has joined #tryton17:33
-!- paepke( has joined #tryton18:20
zodmansomeone knows if trytond depends of genshi ?18:28
cedkzodman: it depends because it depends of relatorio which dependsof genshi18:29
zodmanok ok18:29
zodmancedk: tnx i updating to last tryton on foresight linux :)18:29
cedkzodman: indeed we use genshi filter in report.py18:29
zodmanok ok np!18:30
cedkzodman: dependencies are in the setup.py18:30
zodmanok cedk foresightlinux detect dependencies automatic. Ok i take a look the tnx!18:31
-!- FWiesing( has joined #tryton18:54
udononicoe: It seems that tryton-users.svg is missing on tip?18:55
nicoedid you update to the last version ?18:56
nicoeCedric made a patch renaming it a few days ago18:56
nicoeudono: it has been renamed to tryton-party18:58
udononicoe: I thought I did it. Let me check...18:59
udononicoe: ugs... forget to hg up ...19:00
udononicoe: thanks!19:00
nicoeyou're welcome19:00
-!- nicoe_( has joined #tryton19:48
-!- plantian( has joined #tryton19:49
-!- Timitos(~kp@ has left #tryton20:39
-!- Mayank(~beakkon@ has joined #tryton21:23
-!- sharoon( has joined #tryton21:52
sharoonhi everybody, we are working on the e-commerce problem and we are hitting the "unknown unknowns" issue and our alternatives are the infamous EAV or Schemaless NoSQL databases21:54
cedksharoon: what is "unknown unknowns" issue?21:56
sharooni remember a discussion in the IRC between cedk paepke bechamel yangoon on using MongoDB / CouchDB as a backend for tryton21:56
sharooncedk: consider the example of a company which sells books and furniture21:57
sharooncedk: the attributes on product for books are going to be different from the attributes on furniture21:57
sharooncedk: this bloats to several thousand attributes when you have many different kind of attributes21:58
sharooncedk: the possible ways to implement this are:22:01
sharoon1. Keep expanding the table by _inherit (slows downt he table and limitation of 1600 columns)22:01
sharoon2. Use _inherits - same issues as above and more cumbersome to manage views and will introduce lot of joins and nested _inherits is not supported22:01
sharoon3. Use EAV (Entity attribute value) - Difficult to manage, SQL joins etc22:01
sharoon4. Use two storages. The main data in product.product and all supplementary as schemaless object in a NoSQL database22:01
cedksharoon: I don't trust in EAV model22:02
sharooncedk: Glad that you dont.... i tried EAV and it sucks ;)22:02
cedksharoon: and I would like to understand what are those attributes?22:02
sharooncedk: we also have example of magento22:02
sharooncedk: Eg. in case of book. ISBN, Editor, Edition, Author, Publisher22:03
cedksharoon: did you read
sharooncedk: reading now22:03
cedksharoon: for me your example must be solved by creating a new Model that inherit product22:04
cedkI really trust in schema :-)22:05
sharooncedk: but imagine the same company sells cameras, the new attributes will be like BatteriesIncluded, DisplaySize, BatteryType, OpticalSensorResolution, OpticalZoom etc etc22:07
sharoonso as the types of product increases - so will these attributes22:08
cedksharoon: and what is the problem?22:08
sharooncedk: using _inherit - this will keep increasing the no of columns in the DB22:08
cedksharoon: every product that shares the same attributes (properties) will be in their specific table22:08
cedksharoon: managing tables is what DB is about22:09
sharooncedk: that would be using _inherits ?22:09
cedksharoon: yes22:10
sharooncedk: that is possible when the problem is "known unknows" not "unknown unknowns"22:10
cedksharoon: don't understand those quotes22:10
sharooncedk: if you know before hand that the customer is gonna sell cameras and books we can design for both22:11
sharoonthat is known (columns) unknown (values)22:11
sharoonbut often the case is unknown (columns) unknowns (values)22:12
cedksharoon: I don't think in users defining schema22:12
cedkit is the best way to kill performence, logic and rational ...22:13
sharooncedk: its the "hot" feature eg. Magento, Virtuemart22:13
cedksharoon: feature like this are not useful indeed, what can you do with unknonw things?22:14
cedkyou can do nothing as you don't know22:14
cedksharoon: expect perhaps just redisplay it like it was encoded22:14
sharooncedk: hmm, it is what makes magento the popular system it is..... custom attributes is everythng22:15
sharooncedk: for example you may want to compare a product to product22:15
cedkso the simple way will be to store it in a format XML, JSON or whatever in a varchar columns22:15
sharooncedk: a blob ?22:15
cedksharoon: you can not compare "apples and pears" (don't know if it is English :-)22:16
cedksharoon: yes a blob as it is unstructured data22:16
plantian*apples and oranges22:16
sharooncedk: exactly, so if you  need to compare one camera with another camera, you need to do it with its own attributes22:16
sharooncedk: which is why maintaining its values distinctly is required22:17
sharooncedk: and that is where a XML, JSOn storage will fail22:17
cedksharoon: which is why it is better to have it in a table (with schema)22:17
sharooncedk: you will not be able to search like megapixels = 1022:17
plantianI think extensive searching and browsing of products might be outside the realm of Tryton and connecting tryton products with another model outside of tryton might be better.22:18
cedksharoon: the fastest way to solve this search is having a columns megapixels with an index22:19
cedkplantian: not completly sure but it is a possibility22:19
cedkplantian: but I still think it is smarter to have a good database schema22:20
sharooncedk: back to the same question - what happens when the number of such attributes is really high22:20
cedksharoon: it is not a problem as it is in separate tables22:21
sharooncedk: looking at a design doc before me each product type has about 10 attributes each and they will be adding product types in the future22:21
sharooncedk: are you sure you can extend (product.product) using _inherits when it already extends from product.template with _inherits ?22:21
cedksharoon: it should but I think I fill a bug about search22:22
cedksharoon: and even, it is not required to have _inherits22:22
cedksharoon: _inherits is just to show a merge of the 3 tables22:23
sharooncedk: ok, so what is the alternative ?22:23
cedksharoon: the alternative to what?22:23
sharooncedk: u mentioned that it is not required to have _inherits?22:24
cedksharoon: it is just a many2one to product22:24
cedkso you have your products in product_product and you store into the camera table the attributes for camera and just link it to the corresponding product22:25
sharooncedk: yes, this works and is ideal - but any idea on what to do when the column types is also not known prior ?22:26
cedksharoon: blob22:26
sharooncedk: :( search limitations then22:27
cedksharoon: you can not search for unknown22:28
sharooncedk: i really wish i could tell that to the customer :D (but as usual systems like magento does it... )22:29
cedksharoon: and I read many times that magento has big performence issue because of that22:29
sharooncedk: thats because they suck in EAV22:30
sharoondesign pattern22:30
sharooncedk: what I am now thinking is go schemaless with MongoDB22:30
cedksharoon: there is not soo much way to store data: schema, EAV or blob22:30
cedksharoon: MongoDB == blob22:30
plantiancedk: I think it has indexing and searching though right?22:31
cedksharoon: I know it is a shorcut but the way you think you will use MongoDB will be the same as having a blob in PostgreSQL22:32
sharooncedk: plantian: yes it has indexing, search and a lot more of other features22:32
cedkplantian: yes but only the one you (DB manager) create22:32
cedkplantian: with code22:32
cedksharoon: and ?22:36
sharooncedk: plantian: any thoughts ? on a quick googling seems like schemaless DBs are replacing Old School EAV22:36
sharooncedk: this is scary -
plantiancedk, sharoon: I think it is a good compromise to store less important data outside of tryton in order to gain flexibility that Tryton cannot afford to give because it is full ERP.22:38
plantianAlthough then its just one more system to maintain though so it is just another trade off.22:38
sharooni agree, and i think such a schemaless DB could also be used for storage of documents (ir.attachments) which are currently stored in the filesystem22:38
cedksharoon: why ?22:39
plantianha yeah that seems to have nothing to do with this22:39
sharooncedk: GridFS22:39
cedkplease don't fall in this buzz/fanboy trap where everybody wants to put NoSQL everywhere22:39
cedksharoon: until now the best way to store file is a filesystem22:40
plantiansharoon: I agree with cedk's concern.22:40
plantian*about traps22:40
sharooncedk: plantian: and
cedksharoon: I guess that FS of GridFS is for filesystem so no need to change anything to use it22:41
plantiansharoon, cedk: I am interested in this extensible product problem though.  I have been stuck with one type of product for a long time and need to get a real e-commerce solution going with products other than just Plants(the only product I have now).22:42
plantianRight now, party because I moved from other system, I have plant products in another system and references between tryton and that system in order to have search/browse on the website and display product brochure information.22:43
sharooncedk: if there are large (**if**) the auto sharding etc could be of great use was what i thought :) i guess thats the way google big table works ?22:43
cedksharoon: yes flexible sharding is what is missing right now in trytond22:44
cedksharoon: need to find how to solved22:45
cedksharoon: but it should be not too complicated22:45
sharooncedk: one possibility would be making sure that the folders in the first level (from the digest) decide the sharding (based on mounts)22:46
cedksharoon: that's filesystem configuration (no for Tryton)22:47
cedksharoon: I mean the number os subfolder should be dynamic22:47
sharooncedk: got you :) but wont it be transparent to use something like mongodb... it will also take away the pain in the current system in the way we directy try to manage filesystems22:48
cedksharoon: if mongodb provide a filesystem interface22:49
sharooncedk: i guess it does... but I am just starting to use it.22:50
sharooncedk: so coming back to original question - what do you suggest ?22:51
sharooncedk: interesting22:51
sharooncedk: but I am not sure we need that ... we could use the native get and put ?22:52
cedksharoon: for me the best is table shema22:52
cedksharoon: filesystem is an well define API for file22:53
sharooncedk: what do you think of a ModelSchemaless which could be one of the ancestors/bases of a tryton object in join with ModelStorage ?22:53
cedksharoon: so we use it22:53
sharooncedk: i agree abt the file storage.... this is about the attribute problem22:53
cedksharoon: I don't think it is good22:53
cedksharoon: I'm more and more thinking about merging ModelStorage and ModelSQL22:53
cedksharoon: because any other data storage will be outside the transaction which is really bad for "ERP"22:54
sharooncedk: thats not a bad idea though .... we discussed it once22:54
cedkand more over all the Model schema is linked to RDBMS22:54
sharooncedk: agree on that and we cant have any object being purely schemaless22:55
sharooncedk: it could be "Not Only SQL"22:55
sharooncedk: we use Schema for all regular information22:55
sharooncedk: and all extra attributes etc use the Schemaless ?22:55
cedksharoon: so you can not use MongoDB to solve your issue of schemaless data22:56
sharooncedk: MongoDB alone is not a solution, it has to be used along with ModelSQL even if I have to use it22:56
cedksharoon: so I don't see the benefit22:56
cedkwe agree that the paradigm of Tryton is schema ?22:57
sharooncedk: use both systems for what they are good at: ModelSQL for the webll defined schema we have and No(t only)SQL for less defined Schemaless data22:57
sharooncedk: yes22:57
cedksharoon: so you can not use Tryton with schemaless paradigm22:58
sharooncedk: :( I have to search for more solutions then... please do let me know if you find something interesting which could address the problem22:59
cedksharoon: and I repeat a schemaless solution will be the same as storing data in blob22:59
cedkwith just some faclilities for developpers22:59
sharooncedk: i am now considering the Blob also with a function field and allow search like the new search API bechamel proposed23:00
cedksharoon: you can store data in a format that allow to make some search easily23:01
sharooncedk: easiest i guess would be JSON23:02
cedksharoon: like: <attribute name>:<value>,...23:02
cedksharoon: not sure JSON will be the best23:02
sharooncedk: ok23:02
cedksharoon: here you need to store a list of couples23:02
sharooncedk: the value could be struct/list23:03
cedksharoon: by the way, PostgreSQL has a list column type23:03
sharooncedk: checking23:04
cedksharoon: not sure if it is a good idea23:05
sharooncedk: i think i have to try prototypes of both23:05
sharooncedk: also any ideas on how we could have dynamic views without affecting performance ?23:06
sharooncedk: one example we already have is the analytic accounts way of doing it23:06
cedksharoon: yes but at the end it is static view23:07
cedksharoon: the view is only generated on the fly23:07
sharooncedk: yeh true23:07
cedksharoon: doesn't seem that SQLite got array type23:09
cedksharoon: why not a kind of structured text format23:09
sharooncedk: will have to investigate, but would prefer to depend on standards23:10
sharooni am told "oracle" has some kind of XML field which does allow search, xpath for use in such cases23:11
cedksharoon: not sure it will have good perf but any blob will not have23:13
sharooncedk: agree23:13
sharooncedk: i will put whatever i develop to code review and lets see if you find it useful23:14
plantiansharoon: Will you be searching the arbitrary columns of products IN tryton or on e-commerce site?23:14
sharoonplantian: mostly from e-commerce site - like filter all cameras at 10 MP23:15
plantiansharoon: Seems that you wouldn't need/use tryton search capabilities in that case.  Would website interface directly query trytond?23:16
sharoonplantian: yes23:16
sharoonplantian: would be more like, search db.things.find( { x : 3, y : "foo" } ); get the product_id and return all records with the ids in the ids23:18
sharoonreturned by the NoSQL23:18
plantiansharoon: Why not just store all you need in nosql then and not query trytond at all until sales are made ?23:19
sharoonplantian: prices, pricelists, stock availability23:19
plantiansharoon: Yes, I guess I kind of cache those now into another system for website to access.  I am worried about performance problems caused by ERP overhead.23:21
plantiansharoon: Are you going to develop e-commerce from scratch ?23:22
sharoonplantian: i have done that already23:22
sharoonplantian: now its developing advanced features like this one :)23:23
plantiansharoon: Performance is okay?23:23
plantiansharoon: I am interested in solution because I have some mutant solution right now that requires batches of things to be sent back and forth between systems(one of them tryton).23:24
sharoonplantian: its fine, we did some load testing and we avoid direct hits to DB for readonly requirement by use of Cache23:24
sharoonMemcached to be precise23:24
plantiansharoon: How many products and how many inventory locations ?23:25
sharoon1 inventory location and 400 products23:25
sharoonthats small we know23:26
sharoonwe are now on a bigger project with about 30 stores and 200 K products23:26
plantianI think inventory location is hardest part but also dynamic price calculation is difficult.23:26
sharoonplantian: yes, but dynamic calculation must always be cached23:27
plantianI only have two warehouses but each warehouse has many locations inside it.23:27
plantiansharoon: Yeah I use memcached on the site but not for caching product information.  Its good to know you have gotten a direct interface to work, maybe I will try to move everything to trytond one day.23:28
plantiansharoon: I had to add another type of list to determine if products are salable at different stores,  have you needed to do that too ?23:30
sharoonplantian: we implemented a logic where each website could have categories enabled and then selectively disable products if required23:30
plantiansharoon: Nice, I called them salable lists and they are modeled after price list.  I had to add another level of flexibility though and have product store salability and product store pricing to be used as the basis for price lists and salable lists.23:33

Generated by 2.11.0 by Marius Gedminas - find it at!