IRC logs of #tryton for Saturday, 2010-05-22 #tryton log beginning Sat May 22 00:00:02 CEST 2010
-!- udono( has joined #tryton00:24
-!- ikks_(~ikks@ has joined #tryton03:31
-!- rednul(~rednul@ has joined #tryton04:58
-!- rednul(~rednul@ has joined #tryton05:13
-!- yangoon( has joined #tryton05:20
-!- mr_amit(~amit@ has joined #tryton06:27
-!- rednul(~rednul@ has joined #tryton07:37
-!- cedk(~ced@gentoo/developer/cedk) has joined #tryton09:18
-!- enlightx( has joined #tryton09:28
-!- eLBati(~elbati@ has joined #tryton09:57
-!- mr_amit(~amit@ has joined #tryton10:53
-!- Timitos(~timitos@ has joined #tryton11:01
-!- tekknokrat(~lila@ has joined #tryton13:00
-!- mr_amit(~amit@ has joined #tryton13:13
-!- sharoon( has joined #tryton13:48
sharoonhi cedk13:49
cedksharoon: hi13:49
sharooncedk: can we discuss the email blueprint and existing implementation13:50
cedksharoon: first, did you test the trigger implemntation ?13:50
sharooncedk: not yet13:50
cedksharoon: just two minutes, I have a phone call13:50
sharooncedk: ok, i will prepare an agenda now13:50
sharooncedk: 1. Splitting of Module13:55
sharoon2. Accounts configuration - SMTP/IMAP13:55
sharoon2.a) Company Accounts13:55
sharoon2.b) Scheduler/Cron13:55
sharoon3. Mailbox/email queue13:55
sharoon3.a) What goes into core?13:55
sharoon3.b) Scheduler/Cron13:55
sharoon4. Templating13:55
sharoon4.a) Mako + Django13:55
sharoon4.b) Genshi?13:56
sharoon5. Legacy Support?13:56
sharoon6. IMAP4/SMTP - Twisted ?13:56
-!- ikks_(~ikks@ has joined #tryton13:57
cedkACTION back14:00
cedksharoon: I will send an email on mailing list to inform about this talk14:01
sharooncedk: ok, and we schedule a time?14:01
cedksharoon: as you want14:01
cedksharoon: it can be now for me14:01
sharooncedk: ok, we start then after 5 mins from mail14:01
cedksharoon: ok14:02
cedksharoon: send14:05
sharooncedk: just tweeted about it as well14:06
sharooncedk: lets start at 14:10 CEST14:06
cedksharoon: yes it is in the email14:07
sharooncedk: cool14:07
cedkok at my clock it is 14:1014:11
sharooncedk: i guess its time and we start?14:11
sharoon 1. Splitting of Module14:11
cedkI think we should make modules that do one thing14:12
sharoonso the module needs to be split and we need to decide what goes into what14:12
sharoonand possible sensible names too14:12
cedklike that we can change the behavior easily because you must always think about how to customize the code14:12
cedkalso it is simplier to develop14:13
cedkand to test with unittest14:13
sharoonok so what are the modules14:13
sharoon1. email_queue (core)14:13
sharoon2. email_configuration14:14
sharoon3. email_template14:14
cedkfor me there is at least 4 concept in poweremail14:14
cedk- the storage of email14:15
cedklike a maildir14:15
cedkthe storage must be linked to res.user14:15
sharooncan you explain?14:15
cedkso there is folders (or tags) linked to a user (or many) and emails are linked to one or many folders14:16
cedkand folders can have a tree structure14:16
sharoonfolders == IMAP folders ?14:16
cedkno necessary, we must not think about protocols now14:17
cedkit could have some special folders14:17
cedklike a folder with sended emails14:17
cedkor a folder with new emails14:18
sharooni think these could be the only defaults14:18
sharoonrest is user defined14:18
cedkI'm not sure we must fixed the names14:18
sharoonit might be necessary because only mails in queue and with folder outbox will be sent by the cron job14:19
cedkI don't think we need of Archive, Outbox nor Drafts14:19
cedkI'm not agree, this is not how emails are design14:20
sharooncan you explain?14:20
cedksending email is an other thing and is not linked to the user's folders14:20
cedkthat is why you must define POP/IMAP accounts and also SMTP14:21
cedkin email clients14:21
cedkand in this module we have only storage functionality14:21
cedkso it will have only INBOX14:21
cedkbecause to store we need to have a default folder14:22
cedkthat is all for the first concept14:22
cedk- second concept: email sending14:23
cedksending email means queuing the emails and send it14:23
cedkall emails goes into the same queue14:23
cedkfor the sending, there is 2 possibilities:14:24
-!- paepke( has joined #tryton14:24
cedk- it forward all to a real email server like postfix, sendmail etc.14:24
cedk- it connect to the dest server and deliver it14:25
cedkI think the second one is a complex task14:25
sharoonsecond is complex14:25
cedkbut we should design the software to be able to do both14:25
sharooni think for first version leave only option 114:25
sharoonbut provision for 2 is left14:25
cedksharoon: agree14:26
sharoonok, now to the design14:26
cedkand Tryton has already configuration for a smtp connection14:26
sharoonah! thats something that needs review14:27
sharooni think it should be possible for defining email accounts for each database14:27
sharoonand for each user14:27
sharoonif he wishes to14:27
cedksharoon: what do you call email accounts?14:27
sharooncedk: SMTP/IMAP/POP3 connections14:27
cedkyou must not handle SMTP with IMAP/POP14:28
sharooncedk: nope14:28
cedkthis is different purpose14:28
sharoonSMTP and IMAP/POP314:28
cedkI think Tryton must choose to always send email with SMTP14:28
sharoonyes, only SMTP is planned to be built14:29
sharoonbut are we talkng about the builtin SMTP?14:29
sharoonthats set in the config?14:29
sharooni think that may not suffice14:29
cedkI think it is enough14:29
sharoonReason: the tryton server gets the SMTP settings14:30
sharoonnot the database14:30
cedksharoon: what is the case you think it is not enough, I will explain how to do it :-)14:30
sharoonso if a Tryton using organisation has multiple companies in separate databases but same server then it will not be possible to use their preferred acoount14:30
sharoonfor example: you have both and address14:31
sharoonand assume that both are two companies14:31
sharoonif you want to use these accounts then you need to have two instances of server right?14:31
cedksharoon: what do you name accounts ?14:32
sharoonACTION am i missing something14:32
sharoonaccounts are email accounts14:32
cedksharoon: what is email accounts ? :-)14:32 and are two accounts14:32
cedksharoon: no it is 2 addresses14:32
cedkI think you want to express the fact that you want to use 2 different SMTP outgoing server per domain14:33
cedkam I right14:33
sharooncedk: lets use a common tool for reference14:33
sharooncedk: lets use the lingua of evolution or thunderbird14:33
sharooncedk: whichever u prefer because it makes it easy and common for everybody (even non techies) to understand14:34
cedksharoon: if you want but both application doesn't show you how emails are processed14:34
sharooncedk: that was for the lingua14:35
sharooncedk: like 'accounts' and 'addresses'14:35
cedkcommon email configuration in companies are one domain with one smtp server14:35
sharooncedk: that suffices for common job like... in django we use it for sending crash reports etc14:35
sharooncedk: but to actually send emails companies and users prefer to use either personal accounts14:36
cedksharoon: if you refer to thunderbird accounts is only IMAP/POP and SMTP is a separated configuration14:36
sharoonACTION thunderbird is now the official reference for lingua14:36
cedksharoon: I still don't understand why you talk about personal accounts for email sending14:36
cedksharoon: you could have authentication for SMTP connection but that is all14:37
sharooncedk: in our company, we have two people for support14:37
cedkyou don't require to have an 'email account' (thunderbird def) to send emails14:38
cedksharoon: ok, and I'm pretty sure that they use the same SMTP server14:39
sharoonyes, but with authentication14:39
cedksharoon: because they connect to it from PC, but here it is the Tryton server that will connect to it and that need to be authenticated14:39
sharooncedk: but two different domain?14:40
cedksharoon: that is not a problem with a good configuration of the smtp server14:41
sharooncan you explain? how do you configure smtp to talk to two different servers?14:41
cedksharoon: you must know that the FROM in email is not linked to the authentication on the SMTP server14:41
cedksharoon: don't understand last question14:41
sharooncedk: for example assume that a company uses google apps (like we use)14:42
cedksharoon: bad :-)14:43
sharooncedk: then one db will connect to with the authentication of individual user14:43
sharooncedk: agree on the bad ;)14:43
cedksharoon: what is the domain ?14:43
cedksharoon: or custom one ?14:44
paepkecedk, disagree: from in email could be linked to an authenticated user. from could be rewritten or the mail could be dropped. its not common in the wild, but i know some setups.14:44
sharoonthat is the factor with SMTP servers with authentication14:44
cedksharoon: according to dig -t TXT you can send email with FROM * from any server14:44
sharoonACTION disastrous!14:45
cedkpaepke: yes, you can customize your SMTP server to do any thing but this is configuration14:46
cedkpaepke: but you must configure your SMTP server to work with your applications14:46
sharooncedk: when a company owns the SMTP server then its fine14:46
sharooncedk: what if it doesnt? as with many cases14:47
paepkecedk, i agree with sharoon.14:47
cedksharoon: smtp server cost nothing to install on the same host than Tryton14:47
sharooncedk: but that introduces a new change in an organisation for nothing worth it14:48
sharooncedk: and additional headache if they use hosted14:48
sharooncedk: hosted mail service or exchange etc14:49
Timitoscedk: i also agree with sharoon14:49
cedkok, I will explain my big plan :-)14:50
cedkmy vision of emails in Tryton is:14:51
cedkto store all user emails14:51
cedkbut not send it14:51
cedksending will be done by the email client of the user14:51
cedk(with his own "email account"14:52
cedkso Tryton doesn't require to know the "email account" of users as it will not send emails14:52
cedkthis way, we will have all the emails of a company stored in Tryton then we could process it14:53
sharooncedk: i think it will be bad design because, it has the current problem with the Open ERP thunderbird interface.... when everything becomes hosted & cloud based, we should not base designs on applications like a desktop email client. Also this has disadvantages when a web client is used and access could be from anywhere?14:53
cedklike link it to party (for crm case)14:53
cedksharoon: if you want email client in web interface then you can go for software like horde etc.14:54
Timitoscedk: what about email that will be created by the tryton server like a confirmation email?14:54
cedksharoon: and I don't say to use something like thunderbird OpenERP interface14:55
cedksharoon: it will work out of the box for any email client (with IMAP support)14:55
paepkecedk, so sending an invoice via email would mean that the user has to be in front of the computer and pressing the "send" button in tryton to pop up the mail client and press there send, too?14:55
cedkTimitos: then it is an email sended by the Tryton server so it will use Tryton server account14:55
cedkTimitos: define in the configuration14:55
-!- saxa( has joined #tryton14:56
sharooncedk: but the question remain in such a design: how do you send from different accounts14:56
cedkpaepke: it depends of what you name sending an invoice via email14:56
sharooncedk: that is currently possible with the print and email as well right?14:56
cedkpaepke: if you talk about an automatic process then it is the server jobs14:56
Timitoscedk: what about a multidatabase solution? maybe i do need two different mailservers.14:56
sharooncedk: what Timitos has said is according to me the biggest bad in such a design14:57
cedkpaepke: if you talk about salesman sending an invoice then it is with the salesman email client14:57
sharooncedk: what about automatic emails?14:57
cedksharoon: print-send yes14:57
sharooncedk: it actually becomes a manual email then?14:57
cedkTimitos: no one server can be configured to handle both14:57
cedksharoon: automatic emails are sent by Tryton server with Tryton server account14:58
paepkecedk, yes (salesmen). but its preferrable to have even in automated invoices the sender adress of the responsible salesman. not a standard no-reply@tryton.local14:58
sharooncedk: thats a major major requirement!14:59
cedkpaepke: yes but it can be set with the X-REPLY-TO14:59
cedksharoon: what?14:59
sharooncedk: can you explain how it happens in the case of some service like exchange or google apps15:00
cedksharoon: ok, but first the goal is to replace such services15:00
Timitoscedk: there are cases where such configurations with one server for different domains make the risk to see the email as spam.15:00
sharoonTimitos: agree on that! i had faced the issue15:01
cedkTimitos: no, you must configure your domain correctly15:01
sharooncedk: especially google treats it as spam15:01
cedksharoon: give me the example and I will give you how to deal with it15:01
Timitoscedk: there a configurations on the net that do not allow this. many small companies will be not able to do these configurations15:01
cedksharoon: dig -t TXT b2ck.com15:01
cedkTimitos: I'm not sure15:02
Timitoscedk: not every provider allows configration of MX-Records15:02
cedkTimitos: if they own an domain15:02
paepkecedk, does that x-reply-to overwrite the "from" in the mail client? i don't think so. it will look disgusting to the recipient.15:02
sharooncedk: the issue if we design assuming the companies which are going to use tryton to be this complex using their own mail servers and configuring them for a typical 5-10 user company?15:02
cedkTimitos: it is not the MX records15:02
cedkpaepke: reply-to will make the reply goes to this address15:03
sharooncedk: i guess every company will need people with ur know-how and knowledge to use the system15:03
sharooncedk: i guess we should make it as simple as possible15:03
Timitoscedk: there are also other problems15:03
paepkecedk, yes, i know how that works, but it will not look like the mail was from the guy is responsible for you. it will be from an automated system. not good.15:04
cedksharoon: it will work for almost all configurations15:04
sharooncedk: but it needs a genius like u to configure ;)15:04
cedksharoon: no15:04
cedksharoon: did your internet provide give you a SMTP server ?15:05
cedkpaepke: the best is to setup the FROM with the right addresses, but you talked about a special configuration where the SMTP server will rewrite the FROM15:05
sharooncedk: no!15:06
sharooni use google apps because it makes life easy and my company everybody uses android!15:06
sharoonam sure there are many companies out there who do that too15:06
sharoonno small size business (non IT) company will have such a setup15:06
sharoonthey might already have something like hosted outlook, google apps or something like that15:07
cedksharoon: I'm pretty sure the google doesn't override the FROM in SMTP server15:07
sharooncedk: you get to send a mail using that user only if u login to using the username and password15:07
cedksharoon: yes but you can create a user on google for Tryton15:08
cedkother thing important: if you want Tryton store account information for all users then you must store password in clear in the database15:09
cedkthis is a really big security issue15:09
cedkone place with all password of every body15:09
Timitosi do not think that it is possible to store passwords of all users. but i think it will be needed to define smtp servers by database15:11
cedkTimitos: why ?15:12
Timitoscedk: FYI: if you mean SPF.  there are also many providers that will not allow any configuration of this. and these are especially the big ones15:12
sharooncedk: password access by read should be crapped15:12
sharooncedk: password access should be allowed only by browse15:13
cedkTimitos: dyndns for 15$/year15:13
Timitosand many small companies are hosting their domains there15:13
cedksharoon: this is not the issue15:13
sharooncedk: we cant ask users to change email accounts they used for years15:13
cedksharoon: storing clear text password in database is bad15:13
sharooncedk: I agree on that15:14
Timitoscedk: i do not see how dyndns should solve this issue15:14
paepkecedk, two other things (have to go for half an hour): sending big mails (like reports, results or pictures) will lead to download the full mail to the client and send it again back to somewhere else. on a slow internet connection this would be horrible for the user.15:14
cedkTimitos: emails are linked to domain15:14
paepkecedk, another thing you should think about is email-filtering on server side. like sieve or procmail.15:14
cedkTimitos: if you transfert your domain to dyndns you will be able to customize dns records15:14
sharooncedk: also the idea becomes impossible in future where we talk about hosted web client and mobile client15:14
cedkpaepke: you assume that Tryton server is on internet with a slow connection how could you work on that15:15
Timitoscedk: yes. but i cannot let our customers wait for 1 year to get a working solution because they cannot transfer their domain because of strict contracts15:15
cedkpaepke: agree on email filtering by will come in other topics15:16
sharooncedk: its not a good idea to have too many changes15:16
cedkTimitos: leave the customer15:16
sharooncedk: according to gartner... changes cause 60% failures in information system implementations15:16
cedkTimitos: you will have always bad stuffs with such customer15:16
sharooncedk: every user to organisation will be resistant to change15:16
cedksharoon: excactly, my design will change any thing to users bahavior15:17
sharooncedk: i dont think its a global solution because more and more users are shifting from desktop based solutions to hosted apps15:18
cedkTimitos: and I think you can transfert domain but you will still need to pay the previous one for one year15:18
Timitoscedk: i know a provider where this will be not really possible.15:18
cedksharoon: ok, if you give your information to outside and don't want to change how could you get it  back15:18
cedkTimitos: that is a bad provider and should not work with it15:19
Timitoscedk: but i understand your strategy better now.15:19
sharoonOk i think i have a solution to this problem15:19
Timitoscedk: yes. if i would have known the customers when they did their decision this is clear15:19
sharoonACTION One possible solution15:19
Timitoscedk: but unfortunately there are some other people that do things i would not do15:20
cedkTimitos: you can do nothing with such customer until he leave his previous IT provider15:20
sharoonlet tryton just like now have the default emailing SMTP feature and the mailbox15:20
sharoonthats just what we have now15:20
sharoonbut, lets have a separate module where the user can configure his own accounts and the email queue will be overridden by that mdoule15:21
Timitoscedk: i cannot always wait for the perfect customer ;-) but maybe i can help some to be.15:21
cedksharoon: for you google issue, you can still put a postfix on Tryton host, and write transfert rule base on from to relay to google smtp with the right account15:22
sharooncedk: Greek & Latin15:22
sharooncedk: and we ususally have painful administrators who are third party IT support people who will screw our implementatin for nothing if we say this15:22
cedkTimitos: yes, but you can not sale some thing that you can not do15:22
sharooncedk: i guess tryton should not be a deciding factor for the email service etc15:23
sharooncedk: already when there are popular protocols like IMAP/POP3 & SMTP to do anythign we want in email15:23
cedksharoon: it is not required to use email with Tryton15:23
Timitoscedk: only when your concept of tryton is too strict to 'the only right way'15:23
cedkTimitos: there is always a solution15:24
sharooncedk: I think we should assume .... We dont get a text book customer who has done everything right15:24
sharooncedk: if he has everything right then he doesnt need us15:24
Timitosso i pointed out my opinion on this. it is all i can do.15:24
sharooncedk: Shall we have a vote on this point15:25
cedksharoon: if you want but there will be vote on some point15:25
sharoonProposition 1:15:25
cedklike clear text password storage15:25
sharooncedk, whether we store password in a db or python file, it has the same effect according to me15:26
sharoonsomebody who has access to server has both15:26
cedksharoon: yes but only for one account not for accounts of every employee15:26
cedksharoon: especially the google account where I suppose you also store docs and sheets15:27
sharooncedk: if you want to send mail from employee account then username and password MUST be stored somewhere15:27
cedksharoon: perhaps also google apps etc.15:27
cedksharoon: and I don't want to send email from employee15:27
sharooncedk: i think its more of an organisational choice15:28
cedksharoon: no employees must send their own emails15:28
Timitossharoon: is there no possibility to send emails from employee with authentication of another? like a company account?15:28
sharooncedk: thats why organisations have emails for employees! i think we should make it flexible for the company which has their policies15:28
sharooncedk: Timitos: its possible15:29
cedksharoon: otherwise you have the case where employee receive email answer to email that he doesn't know he sended15:29
sharoonTimitos: my argument is: the same tradeoff exists  if password is stored in DB or py file15:29
sharooncedk: in your case where employees dont need emails only configure one account15:29
sharoonsame tradeoff15:30
sharoononly one account15:30
sharoonaccessible to all15:30
Timitossharoon: yes. but maybe it is not needed to store all accounts15:30
sharoonTimitos: agree15:30
sharoonits not needed to store all accounts15:30
cedkonce again, sending email is not linked to email account !!!15:30
sharooncedk: IT IS for many accounts like the google apps and hosted exchange15:31
Timitoscedk: yes. i know. but it dependes on the configuration of the provider. some will reject emails that are authenticated by another account15:31
sharooncedk: we should allow user to use what he already has.15:32
cedkOk we will never have something that works with all crappy configuration15:32
cedkand we must not encourage bad design15:32
sharoongoogle apps is not bad design (though it is)15:33
sharoonwe cant say google is wrong?15:33
cedksharoon: I don't say that15:33
Timitoscedk: sorry. but a configuration like hosted exchange and google apps should be not seen as crappy. these configurations are fact and we should be able to handle them15:33
cedksharoon: it is not required to use smtp of google to send your emails15:33
sharooncedk: can you explain?15:33
cedkTimitos: I don't talk about that15:33
Timitoscedk: why not?15:34
Timitoscedk: why do you not talk about that?15:34
cedkTimitos: bad design is domain that you can not customize15:34
cedkI will make a brief history of email15:35
cedkSMTP is a protocol that allow server to communicate and exchange emails15:35
Timitoscedk: but you cannot be always against others. if we will only do it one way there will be many potential customers we will be not able to serve15:36
cedkany server can communicate with any server15:36
Timitoscedk: i know smtp. this is not the problem. the problem is how it is used today15:36
cedkTimitos: it is used the same way15:36
Timitoscedk: but with some restrictions in many cases15:37
cedkTimitos: no15:37
Timitoscedk: and i do not think that it is good not to look on how it is used today15:37
sharooncedk: "Modern SMTP servers typically use a client's credentials (authentication) rather than a client's location (IP address), to determine whether it is eligible to relay e-mail."15:37
sharooncedk: says wikipedia15:37
cedksharoon: that is not the subject15:38
Timitoscedk: ok. so there no more discussion necessary as you have your point and we will be not able to find a conclusion except yours15:38
cedkTimitos: no, I think you don't know how emails work15:38
sharooncedk: to be frank, i think all my existing customers who are funding the email module needs google apps (and they wont change) and they need sending from company mail accounts from two different databses15:39
sharooncedk: so i think i will have to build a separate module to meet that15:39
cedksharoon: I already said this is possible with one smtp15:40
sharoonwhile the standard parts that need to go into official base may accomodate this design of best practise u propose15:40
Timitoscedk: i understand the configuration you suggest. the only thing i do not agree is that it will always work.15:40
cedkTimitos: only if you have access to your domain15:40
cedkTimitos: which must be something that you own15:40
cedkTimitos: otherwise you have been steal15:41
Timitoscedk: yes. but sorry for this. this is for small companies not always the fact because they made mistakes in past.15:41
Timitoscedk: they own it but they do not have all possibilites of configuration15:41
cedkTimitos: from the web interface, but I'm pretty sure with a phone call you can have what you want15:42
Timitoscedk: no. i know that15:42
sharooncedk: can u explain how u will handle my case?15:42
cedksharoon: setup an smtp server on the Tryton server that will send emails15:42
sharooncedk: lol, that doesnt show in the sent mail history or search results of my client's mailbox15:43
sharoonin google apps15:44
cedksharoon: mail history is not set by the smtp server but by the email client15:44
sharooncedk: if the mail gets sent through the email account, only then it gets logged in sent mail15:45
Timitossharoon: i think you will need to do base modules without the possibility to send throught the employee accounts and to put another module on top to add this possibility.15:46
sharoonTimitos: +115:46
sharooncedk: i think that's the best design15:46
Timitossharoon: the base ones will be part of the tryton framework (i hope so) and the other one will be a custom one15:47
sharoontryton will then stick to best practices while still allowing the user to use his config but at the risk15:47
Timitossharoon: yes15:47
cedksharoon: did you receive the email I send with your email addresses ?15:48
sharooncedk: I guess that ok15:48
sharooncedk: it went to spam15:48
cedksharoon: by the way, as emails are stored in Tryton and Tryton send it, he can put it in the send folder15:49
cedksharoon: of course because DNS is not completly well configured15:49
cedksharoon: but it is doable15:49
sharooncedk: advice me how to block that though15:49
cedksharoon: and also it is suspicious to have email with same from and to15:49
cedksharoon: ok after the talk15:50
sharoonok, next topic15:50
cedkbut the demonstration was that it is possible to send email linked to gmail from other server15:50
cedkwithout having the account informations15:51
sharoonbut cedk it gave me this link instead of ur mail15:51
sharoonACTION conclusions so far15:52
cedksharoon: ok this is because I send it with same from and to15:52
cedksharoon: I could do it with an other email addresses if you want15:52
sharoonACTION Tryton will have an email queue built into the server to which emails can be pushed with all required header tags15:52
sharoonACTION tryton builtin SMTP will push emails from this15:53
cedksharoon: not builtin SMTP15:53
sharooncedk: ?15:53
sharoonbuiltin SMTP client15:53
cedksharoon: queue will be empty by using configured smtp server15:54
cedksharoon: for me builtin SMTP means writing smtp server15:54
sharoonanyway and a custom module will be implemented for the purpose of custom configs with clear warning of security15:54
sharoonusing the custom module would mean disabling the builtin SMTP "client" in python of tryton and using custom configs to push mails15:55
sharoonACTION next topic is templating15:55
cedkI think we should use genshi15:56
cedkbecause already use in relatorio15:56
sharoonI agree on that because its already a dependency15:56
cedkso it doesn't bring new dependency15:56
sharoonbut the bigger picture is how to integrate with triggers because templates are what decides the triggers as well15:57
cedkand if the code is well writen, it will be possible to add other templating with external modules15:57
sharooncedk: agree on that15:57
cedkbefore talking of trigger, I want to talk about template storage15:58
cedkI think template email must be stored with the same model than email15:59
cedkbecause it will allow to design email with the email client15:59
sharooncedk: not really sure16:00
sharooncedk: because how do we handle reports as attachments16:00
Timitoscedk: can you explain this a little bit more?16:00
cedksharoon: attachments can be added on the fly16:00
sharooncedk: in automatic?16:01
sharooncedk: when PO is confirmed send automatic mail to x-user16:01
cedkthere is two things:16:02
cedk- the template16:02
cedk- the rule16:02
sharooncedk: the rule is handled by trigger16:02
cedkthe template is just the email data16:03
cedksharoon: why not16:03
cedksharoon: ok agree, the module must extend ir.trigger16:04
sharooncedk: no16:04
cedksharoon: why?16:04
sharoonthe trigger when triggered calls the template render method for the corresponding ID and the mail is rendered for the corresponding active id given by trigger16:04
sharoonthe rendered mail is pushed into the queue and the process is done16:05
cedksharoon: yes16:05
sharoonthe render includes16:06
sharoon - rendering the email id: ro, cc, bcc, subject, mail body16:06
sharoon - multipart objects - attachments16:06
sharoonin manual mode the render is done and the create mail is shown to the user16:06
cedkACTION still agree16:06
sharoonso the user can modify the mail...16:07
sharoonin the list view it should be possible to create multiple emails as well16:08
sharoonselect 5 invoices and action->Send reminder16:08
sharoonthe mails should be generated and pushed to queue16:08
cedksharoon: the example is not very good, because an report will do the work16:09
-!- tekknokrat(~lila@ has joined #tryton16:09
sharooncedk: can you explain16:09
cedksharoon: create a report reminder on invoice, and right click on the report button to send as attachment16:09
sharooncedk: fine... but the point i am trying to make is the templates have to be a separate model where the values have to be stored16:10
cedksharoon: don't understand16:11
cedktemplate must store nothing16:11
sharooncedk:  please explain16:11
cedktemplate is a template16:11
cedkI don't know how to explain what is a template16:11
sharoonok let me do that16:12
cedkso it is an email where some part (define by markup) must be substitued when rendered16:12
cedklike odt file of report are template to generate odt file16:13
sharoona template is some templating engine (genshi) based tool is used to generate the email16:13
sharoon- for example subject - it could be Your order ${object.reference} has been processed16:13
-!- tekknokrat(~lila@ has joined #tryton16:13
sharoon- example mail body16:13
sharoonHello ${ or 'Customer'},16:14
sharoonWe thank you for your order.16:14
sharoonso rendering the template will first render these16:15
sharoonthen it will generate the reports that were to be created16:15
sharoonand add it to the mime/multipart message16:15
sharoonand the email.message object is passed on to the create method of email queue16:15
cedkbut what is the reason to not store template as an email16:17
sharoonit will have lot many more fields than an email would have16:17
sharoonemail will be a subset of template16:18
cedksharoon: which one?16:18
sharooncedk: example: a field builder based fields, report many2many, allowed users, attached trigeer ets16:19
cedksharoon: it is on the rule16:19
sharooncedk: field builder?16:19
cedkit is the same desing then for report16:20 and odt files16:20
sharooncedk: kind of16:20
cedksharoon: field builder is because you think about writing template in Tryton client, but for me it must be done on email client16:20
sharoonbut with extra fields for to, cc, bcc, subject, reply to etc16:21
cedksharoon: from, to, cc, etc are in emails16:21
sharooncedk: and how do you do it in email client?16:21
cedksharoon: click new email, write it and save it in special folder16:21
sharooncedk: thats again on a local desktop16:22
cedkyou will have a real email editor16:22
cedksharoon: go on your web email client, click new email, write it and save it in special folder16:22
sharooncedk: got u16:23
cedkit is the same as email folders are in Tryton16:23
sharooncedk: and how do you attach it to triggers?16:23
cedksharoon: assume that ir.trigger have some more fields added by email template module16:24
cedksharoon: so it will have a many2one to an email stored (flagged) as a template16:24
cedkbut I don't know yet if it must be extended or inherited16:25
sharooncedk: got it, but it should have one more filter that the trigger model and template model must be same?16:25
cedkwith inherit we will be able to create a custom view16:25
sharooncedk: i think inherit is better16:26
sharooncedk: i guess inheriting email will also be necessary16:26
cedksharoon: why?16:26
sharooncedk: normal emails in queue have no model?16:26
sharooncedk: no template builder either16:27
-!- udono( has joined #tryton16:27
cedksharoon: don't understand why you talk about email queue16:28
cedkmodel of template will be define on rule16:29
sharoonthere will many templates in the template folder16:30
cedkir.trigger inherited -> email.rule with all fields you had in poweremail16:30
cedk+ a many2one -> email16:30
cedksharoon: yes, you must choose the one you want16:31
sharoonfine.. looks good16:31
cedkperhaps we could create subfolder by email.rule16:31
sharoonand whats the structure of email.queue?16:31
cedkparams of
sharoonand how do you record if an email has been sent?16:34
cedk+ state field16:35
cedkor better active boolean16:35
sharoonand how do we relate email.queue to email16:35
cedksharoon: we don't16:36
cedkwe should not implement IMAP sending email16:36
sharoonfine... sounds ok so far16:37
sharoonACTION next topic: Email reception16:37
cedksharoon: normally, it was legacy support16:37
sharooncedk: missed it in the accounts section16:37
cedkso for me, Tryton must become the storage place of emails16:38
sharooncedk: It should be possible to recieve mails, store them in tryton and link them to other objects like attachments16:39
cedkthe best way is to configure the smtp reciever server to store emails in Tryton16:39
cedkfor that a procmail like script is ok16:39
sharooncedk: please!!!! dont get on to modifying servers16:39
cedkI'm speaking of the best practice16:40
sharooncedk: for that you have to keep modifying code everytime16:40
sharooncedk: best practice : No doubt16:40
sharooncedk: usability: big questions16:40
cedkTryton should promote best practice16:40
sharooncedk: should promote... not constrain16:41
cedksharoon: there is no constraint16:41
sharooncan you explain how it will work with procmail16:41
cedksharoon: it is not with procmail but with a python script that does the same job as procmail16:42
sharooncedk: thats the same what i said but the script should not be hardcoded16:42
cedksmtp servers need to have a way to store emails16:42
sharooncedk: forget servers, i am talking about fetching emails by POP3 or IMAP16:42
cedkfor POP this will be the same16:43
sharooncedk let me explain my idea16:43
cedkjust instead of having postfix calling procmail like script, it will be an other script that fecth email16:43
cedkfor IMAP, I don't see the goals as IMAP by default leave emails on the server which is not what we want as Tryton is the default storage place for email16:44
sharoonwhats the problem in leaving mails in server?16:45
sharooni dont understand16:45
cedksharoon: duplicated data16:45
sharoonwell IMAP was designed for that purpose16:45
cedkexcept if you will write a synchronize script16:45
sharooncedk: not needed16:45
sharooncedk: IMAP gives read unread info and servers maintain ID of mails16:46
cedksharoon: the purpose of IMAP is to leave email on the server side and fetch it on demand16:46
sharooncedk: i already implemented this for poweremail and it works. i see no reason it shoudl not work here16:46
cedksharoon: synchronisation because if you move emails or read it then you should push back the info to the IMAP server16:47
sharooncedk: its possible to do that, say on read16:47
cedksharoon: first, you must not set email as readed as soon as the real user did not read it16:48
cedksharoon: but it can work if you use IMAP as POP16:48
cedksharoon: fetchone and remove/leave it16:48
sharooncedk: ok, thats upto the user16:49
sharoonlets not talk protocols, it more important to first set the reception16:49
sharoonemails are stored within the email16:49
cedkso the procmail script will create an email record16:49
sharooni dont understand the need of an external procmail script16:49
cedkbecause it is postfix that call it16:50
cedkit is better to have push then pull16:50
sharooni dont know, then this will also be an extra addon16:51
sharoonbecause hardcoding to some script which you then set as cron in the system environment according to me is extra pain while not needed16:51
sharooni dont see a bad practice in fetching mail with the python library16:52
cedksharoon: I don't say that16:52
cedkI said it is better to have push the pull16:52
cedkso you cron example is a pull design16:52
cedkwith postfix it will be a push design because script will be called at email reception16:53
sharoonwhy is pull a bad design?16:54
cedkfor POP, I don't know well the protocol so it will perhaps be a pull in a cron16:54
sharoonwith postfix how do you set the usernames password etc16:54
sharoonthats too much configuration16:54
cedkfor IMAP, you can keep the connection up and server will push you new emails16:54
cedksharoon: which username and password?16:55
sharooncredentials to fetch emails16:56
cedksharoon: postfix no credentials16:56
sharoongoogle apps?16:56
cedksharoon: with POP and IMAP, of course it is required16:56
cedksharoon: but you can not do in any otherway except if you can have a master user/password16:56
sharooncedk: that is only possible for your kind of ppl... not simple users and organisations16:57
cedkagain here is the best practice, you could write your own module that will pull emails16:57
sharooncedk: so that concludes it16:57
sharoonACTION legacy  support16:57
cedksharoon: email object need to have a generic income method that any pull/push code will call to store new emails16:58
cedkthis method just need to get the data of the email16:58
-!- eLBati(~elbati@ has joined #tryton16:59
cedkit will parse the to, cc value to create all the emails and put it into the INBOX16:59
sharooncedk: i think email also should have in addition to create a create_from_email where an email/message as received can be passed16:59
sharoonthe function should parse the mail and read it16:59
sharoonso we stick to the standards16:59
cedksharoon: yes that is the income method I told16:59
cedkbut here is the big question:17:00
cedkhow do we store emails with many to or cc17:00
cedkshall we create one email per addresses or only one email17:00
sharoonchar fields have no limit right?17:01
sharoonso no issue17:01
cedkbut I don't see the link with the question17:01
sharooncollon or comma separated17:01
cedksharoon: I think you don't understand the question17:01
sharoonACTION absent minded!17:02
sharooncedk: please say17:02
sharooni missed the point i guess17:02
cedkyou receive an email with many to receipt (you and your colleague)17:02
cedkso how Tryton will stire this email17:02
cedkone copy per user ?17:02
cedkor one copy and link it to INBOX of each one17:03
sharoonif my account is configured to recieve mail i receive one17:03
sharoonif he has not configured he doesnt receive any17:03
sharoonif he also has confgured he gets a copy as well17:03
sharoonso its nothing to do with to, cc & bcc17:03
cedksharoon: this is because of your bad design :-) with gmail17:03
cedkbut we can find that we receive duplicate emails17:04
cedksame copy17:04
cedkso if we make some rules to link it to party or crm case etc. we will have duplicate email on those records17:05
sharooncedk: i told you u guys are genius,but there are not many out there and certainly my customers dont employ them ;)17:05
cedkand it will not reflect the reality17:05
sharooncedk: we wont have personal accounts related to crm17:05
sharooncedk: it will be one account like sales@tryton.local that will be there17:05
cedksharoon: customer always send with people in cc17:05
cedksharoon: to be sure to receive it17:06
sharooncedk: but it doesnt generate duplicate records because ny personal mail is not attached to crm17:06
sharooncedk: but sure it creates duplicate emails17:06
cedksharoon: the goal is to have all communication (emails) of the company in one place17:06
cedkwhere we could make data mining17:07
cedkand so duplicates data will make data mining less efficient17:07
sharooncedk: lol, then every account has to be configured with credentials17:07
cedkso I think the income method should check if the same email was not received17:08
sharooncedk: that goes against your initial argument17:08
cedksharoon: I don't understand why you talk about credentials17:08
sharooncedk: normal users will need pull, not push17:08
cedksharoon: normal users doesn't care of pull or push (it is only dev stuffs to keep the server load low)17:09
sharoonanyway, i cannot implement this with popular mail services17:11
cedksharoon: why?17:11
sharoonwhich normal customers seem to use17:11
sharoonits extra mainteinance, and administration costs17:12
sharoonso i think this remains separate17:12
cedksharoon: about what are you talking?17:12
sharooncedk: i dont think a script like procmail is easy to maintain + i dont see the overload that will be there in duplicate mails.. some simple de-duplication could be done, but again that could be future releases17:14
sharoonmy first priority is to get something working17:14
sharoonwithout big pits even if it is not a text book implementation17:14
cedksharoon: procmail, you can go with what you want as at least we have a single point for incoming emails17:15
sharooncedk: so i guess all the points have been covered...17:15
sharooncedk: its not possible to have legacy support and still maintain it with so many changes17:15
sharoonso i drop the idea17:15
cedksharoon: no for me duplicate emails is very important17:15
sharooncedk: but its just part of the create_from_mail... we can improve it after intitial design17:16
cedksharoon: I still don't understand what are you dropping?17:16
sharoonits not a big change anywaty17:16
cedksharoon: it is a big change17:16
cedksharoon: because from that linked email to folders will depend17:17
cedksharoon: and also tracking the state of the email (unread, read, etc.)17:17
sharooncedk: i dont think its a priority for a first working release17:17
cedkif you go in one direction, you could not come back to the other one17:18
sharooncedk: i think we keep some basic guidelines so that the basic design does not hinder change17:18
cedkor the migration script will require too much work than implementing at first place17:18
cedksharoon: yes, and we have not yet define the design of email and folders17:19
sharoon    * email.mailbox17:20
sharoon          o name: char17:20
sharoon          o users: many2many email.mailbox-res.user17:20
sharoon    * email.mailbox-res.user17:20
sharoon          o mailbox: many2one email.mailbox17:20
sharoon          o user: many2one res.user17:20
sharoon          o parent: many2one email.mailbox17:20
sharoon          o subscribed: boolean17:20
sharoon          Unique: mailbox, user17:20
sharoonthat was the original design from blueprint17:20
sharooni guess that will suffice17:20
-!- sharkcz( has joined #tryton17:20
cedksharoon: I think it is not correct17:22
sharooncedk: please propose17:23
cedkI think flags must not be on email17:23
cedkbut on the many2many table that link email to mailbox17:24
cedklike that we can put same email in different mailbox17:24
cedkof diffrent user17:24
cedksharoon: I need to think more17:25
sharooni think i will follow the changes on wiki17:25
cedkI will make a proposition the WE17:25
cedkI need to check all commands of IMAP17:26
cedksharoon: I think last topic is the IMAP server17:26
sharooni think we can discuss that after the first part is implemented17:27
sharoonis that ok?17:27
-!- eLBati(~elbati@ has joined #tryton17:43
paepkecedk, sharoon. i know i'm late. may i give you my 2c about that incoming mail?18:09
cedkpaepke: no problem18:12
paepkei agree with cedk about that incoming mail path. if you pull from a pop/imap server you can use traditional tools like fetchmail which work for several years. if you implement this with python a second time, it will be a hard thing cause of the many different implementations.18:12
paepkewith the pop/imap fetching you have to store the username/passwords for every account you fetch. thats the corresponding security hole which you talk about at the sending email topic. but its a common practise. these username/passwords don't have to be the same like the windows or tryton passwords.18:14
cedkpaepke: yes, but the current design will allow also to have pulling18:15
sharoonpaepke: welcome back18:17
paepkethere should be a message deduplication (store only one message) in tryton. a lot of doubled emails for example mails related to project with a lot of members are floating around. every groupware/email server is looking into that cause of the exploding messages. i'm talking about that servers with a database backend.18:17
paepkecedk, its ok to have both setups. just wanna give you my experience about pulling.18:18
paepkeanother point which is currently not mentioned is the attachment. please allow to store that attachment as a file. and only the message in the database. or only the headers. i missed that discussion.18:20
cedksharoon: model design updated on wiki18:20
sharooncedk: checking18:21
cedkpaepke: take a look at the model design on wiki18:21
sharooncedk: are you retaining the falg design18:22
cedkI propose to store email without header in the data_path like the attachment (so no duplication) and only keep header in record18:22
paepkeemail from data_path means a function where you can store it somewhere? cool design decission18:22
cedksharoon: yes, because I move only the duplicate part in the data_path18:22
cedkpaepke: data_path is a way to store file in directory like it is done in proxy cache18:23
cedkpaepke: and it store only once a file18:23
cedkpaepke: because we used digest to name this file18:23
paepkecedk, good.18:24
sharooncedk: sexy design18:24
udonoI read the log, and found a lot of interesting ideas. But some questions remain. Is there a way to support POP before SMTP which is used by cheap email providers? I guess it could be an additional module/script...18:24
cedkI have let the header on the email record because there is some attributes that are unique even on same emails like "Delivery-To:" and "Originate:"18:24
paepkecedk, but why not store the headers with the file. my intention is to have the email as it was received by tryton. with the digest we can see that it was not mangled.18:25
cedkudono: POP and SMTP are different protocol for different purpose18:25
paepkecedk, pop bevore smtp is a kind of poor mans authentication.18:26
udonocedk:  realy? :-)18:26
cedkudono: is that POP for AUTH on SMTP ?18:27
udonocedk: yes,
cedkudono: not supported by python lib18:27
cedkpaepke: for header, here is a scenario:18:28
cedkone email is sended to two users of Tryton18:28
cedkso we want to store it only once (especially if there is big attchment)18:28
cedkbut they will have a different header18:29
cedkbecause email server will add "Delivery-To:" to each one with his email addresse18:29
udonocedk: k18:29
cedkso the data_path storage will store them as different file18:30
cedkthat is why I store the header in the record and the data content in data_path18:30
cedkpaepke: is it ok for you?18:30
sharooncedk: concept of header looks solid as of now18:31
udonocedk: is the E-Mail storage usable for archiving? That means are all incoming mails stored unchanged forever. Deleting a mail from the users client means 'unlink' the email-resource instead of 'deleting' the resource.18:32
paepkecedk, i have to think about that delivered too. i'm not sure about.18:33
paepkeudono, cedk: it should be possible to change the link to the email in the future. like that archiving udo is talking about. just copy over to a different, slower storage.18:34
cedkudono: if we don't delete email records18:34
paepkei'm just thinking: what about a second imap-mailbox with archived mails. if you delete it in your primary inbox it gets an archive flag and it shows up in your archive inbox. so you can clean your mails and its not a big deal to the server. cron jobs could move the files at night to a secondary storage.18:37
cedkpaepke: I prefer logical delete18:38
paepkecedk, for example in germany you have to remain all emails which you recieve for 10 years. (well thats totally short, there are exceptions and so on.)18:40
paepkecedk, but a user wants a clean inbox.18:41
cedkpaepke: logical delete18:41
paepkehow can the user access the mail after that?18:41
cedkreactive it18:41
paepkecedk, please explain. maybe i misunderstand.18:41
paepkecedk, you mean that struck out in the mail client?18:42
cedkpaepke: no it is active field like any where else18:43
udonopaepke: I guess you do this in Tryton, which will be a kind of a control-center of the companies Emails.18:44
paepkeudono, that breaks up the mail managing interface (eg thunderbird)18:47
udonopaepke: why?18:48
paepkeudono, how would you manage it? there has to be an mail client interface in tryton?18:48
paepkeudono, you have to build up sfolderstructure for example to select the mail.18:49
paepkebut ok, you have a great search interface to the mails.18:50
paepkein my oppinion the mail handling should be one interface - the mail client. for example deleting a mail results in archiving into a second mailbox.18:51
udonopaepke: As far as I understood, a simple and generic interface for e-mails will be provided in Tryton. cedk? sharoon?18:52
paepkecedk,  i have that delivered-to only on my gmail account. every other accout don't us it. youre right. that breaks the single instance storage.18:55
cedkI have updated the model storage to store almost the complete email without some define custom header18:56
cedkpaepke: I don't think you must handle deleted email from email client18:56
cedkpaepke: it is more like a backup so when user realize that he need a deleted email, he will ask to the admin to restore it18:57
paepkecedk, yes, like an trash. an important thing. but the user should have the possibility to restore that mail by himself18:58
paepkecedk, before a cron job gets rid of it after 30 days18:58
paepkecedk, don't you have an use case for an extra email archive? outside of your mailbox.19:00
paepkecedk, it would be great to have that without buying an extra email archive appliance.19:00
cedkpaepke: I find email archive unuseful with IMAP19:02
cedkpaepke: it will be something doable with custom module19:02
cedkagain we should build simple module and extend it with other19:03
paepkecedk, maybe youre right with custom module. its just another big point for a lot of companies.19:03
udonopaepke: since email storage is on file system, you can 'archive' your mails with your file system tools.19:04
paepkecedk, imap is not an archive. its an protocol.:-). it could be a possible access protocol to an archive.19:04
cedkpaepke: I mean as I use IMAP I don't have all my emails on my PC but only those who I want19:05
paepkeudono, the big point in archiving is to store it in a way where you can not modify it. that could be done with tryton so that the user cannot change it.19:05
cedksharoon: is the data model ok for you?19:06
sharooncedk: looks ok to me, will get back to you when we are implementing if any issues are there19:06
paepkecedk, youre an expierienced user. i know not much users which are able to subscribe to certain folders19:07
cedksharoon: ok, but try to publish/coderview each modules as soon as possible19:07
sharooncedk: fine19:08
cedkguys don't forget to review triggers19:09
paepkecedk, i think in a freshly designed email server there should be an archive solution integrated. but thats just my personal thoughts.19:10
paepkecedk, sharoon: its impressive that you do such a big thing like programming a whole webserver.19:11
paepkemailserver... :-/19:11
udonopaepke: Data is always changeable. So you need a kind of automated signature of all incoming and outgoing mails to follow changes. I think the Tryton mail storage is not the right place for doing this, because the headers and contents will be changed if I understand correctly. The tryton storage archive is IMHO more useful for data mining.19:11
paepkeudono, the best point to archive an mail is at the incoming smtp-server.19:12
paepkeudono, but at the discussing with integrated polling its necessary to have it.19:13
cedkudono: the archiving principle of no changeable data is a dream19:13
cedkand I think the data_path is good enough for archiving19:14
cedkwe will only remove headers that are added by your own email server19:14
paepkeudono, cause of the changeble headers i were asking about the full mail as stored file.19:14
cedkand nothing about the content19:14
cedknor the common headers like Date, Received:, Message-ID:, Subject: etc.19:15
paepkecedk, the headers are important, too. but you already mentioned it.19:15
cedkpaepke: but not what we will remove19:15
cedkpaepke: because it is something you add to it (with your server)19:16
paepkecedk, yes. youre totally right.19:16
udonopaepke: The best point to archive Emails afaik for Germany administration is to do it with a proprietary appliance which is directly behind your router or a service on demand.19:16
cedkand not all email server add those headers19:16
cedkudono: I'm pretty sure we will do better then that :-)19:17
paepkecedk, the big and good point is that appending other tags or information to the mail like a future crm module is to have that archived, too. like assigining to a project. so searching on archived/deleted mails would be comfortable19:17
cedkpaepke: we will not alter headers to attch email to records19:17
cedkpaepke: it will be with many2one in database records19:18
paepkecedk, but inside the database19:18
cedkpaepke: yes19:18
udonocedk: Just reflecting the conclusion of some Expert talk about Email archiving in Germany in a last year Linux Journal.19:18
paepkecedk, fully agree19:18
cedkudono: common behavior to create a market to sale unecessary stuffs19:19
cedkso I'm leaving19:19
udonocedk: may be.19:19
cedkACTION bbl19:19
paepkeudono, i agree with cedk. but we can discuss that on the ;-)19:23
udonopaepke: I agree with the governmental requirements for Germany.19:32
paepkeudono, jepp19:33
udonopaepke: and as I said, I do not think that Trytons e-mail storage is the right place for archiving mails in order to these requirements... because the requirements are very nation specific.19:36
-!- FWiesing( has joined #tryton20:39
-!- zodman(~zodman@foresight/developer/zodman) has joined #tryton20:51
-!- zodman(~zodman@foresight/developer/zodman) has joined #tryton22:16
-!- zodman(~zodman@foresight/developer/zodman) has joined #tryton23:03

Generated by 2.11.0 by Marius Gedminas - find it at!