aboutsummaryrefslogtreecommitdiffstats
path: root/cloud_mdir_sync/gmail.py
Commit message (Collapse)AuthorAgeFilesLines
* Log the non-HTTP error exceptionJason Gunthorpe2020-09-251-2/+2
| | | | | | For debugging. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
* cms-oauth: Check the token lifetime before returning itJason Gunthorpe2020-07-171-1/+3
| | | | | | | If the mail functions are not being used then the ticket will get stale, check the timestamp before returning it and refresh if too old. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
* gmail: Fix a spelling error in add_message()Jason Gunthorpe2020-07-101-1/+1
| | | | | | | Not sure when gmail sends a message ID without a labelId stanza, but apparently it happens rarely. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
* OAUTH: Make some sense of the scopesJason Gunthorpe2020-06-221-11/+24
| | | | | | | | With the ability to run as a broker for IMAP/SMTP we can limit the scopes requested based on the configuration. Add a fake _CMS_ protocol that refers to the scopes required to operate internally. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* OAUTH: Add support to get an IMAP OAUTH tokenJason Gunthorpe2020-06-221-3/+3
| | | | | | | Latest mutt can do this for MS and GMail providers, provide support for getting the right scope and some examples how to set it up. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* GMail: Instead of polling each Label, poll the profileJason Gunthorpe2020-06-221-9/+14
| | | | | | | | | The global history_id covers all labels, if we see it change then do a delta query to all the labels to figure things out. This reduces the amount of server traffic if there are multiple labels. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Add debugging to try to find an assertionJason Gunthorpe2020-06-221-0/+3
| | | | | | | | | | | File "/home/jgg/oss/sync/cloud_mdir_sync/maildir.py", line 210, in force_content self._store_msg(msgs[content_hash]) File "/home/jgg/oss/sync/cloud_mdir_sync/maildir.py", line 145, in _store_msg self.msgdb.write_content(cloudmsg.content_hash, msg.fn) File "/home/jgg/oss/sync/cloud_mdir_sync/messages.py", line 279, in write_content assert content_hash in self.file_hashes Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Gmail: Retry on 502 Bad GatewayJason Gunthorpe2020-06-221-0/+1
| | | | | | gmail occasionally returns this. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* OAUTH: Only start one webbrowser tab for concurrent authenticationsJason Gunthorpe2020-06-221-5/+0
| | | | | | | | Starting a browser tab per authentication attempt makes a mess, so long as there is something in auth_redirs then the active browser tab will eventually get redirected to the new auth being added. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* GMail: Stop using requests-oauth and just invoke the POSTs directlyJason Gunthorpe2020-06-221-79/+34
| | | | | | | aio-http is much better, requests seems to get stuck occasionaly. This allows all GMail accounts to acquire access tokens concurrently. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Manage concurrency to avoid running out of file descriptorsJason Gunthorpe2020-05-301-16/+18
| | | | | | | | | | | | AIO does not seem to strictly pipeline work, so it is possible for a very large number of message download tasks to be scheduled. Each one that gets scheduled opens a file descriptor, and so we can run out of them fairly fast. Strictly limit the number of open files per mailbox during message downloading. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Add OAUTH Credential serverJason Gunthorpe2020-05-281-9/+25
| | | | | | | | | | | | | | | | The OAUTH credential server allows CMS to ack as an OAUTH broker and supply bearer tokens to other applications in the system. Currently this only support SMTP tokens for outbound mail delivery. A UNIX domain socket is used to communicate between the SMTP agent and CMS. A simple one line protocol is used to specify the account requested and CMS returns the plain XAOUTH2 response string. The agent is responsible to base64 encode it. This works for GMail and O365 mailboxes. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Remove cfg.domainsJason Gunthorpe2020-05-281-14/+9
| | | | | | | | Directly connect the 'account' API objects to their mailbox users through the config language instead of trying to fix it up after the fact with a dictionary. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* gmail: Fix determining the start_history_idJason Gunthorpe2020-02-241-7/+13
| | | | | | | | | | | | Occasionally gmail returns history records that have junk in them, for instance messages IDs that don't exist. This appears to be a bug. Using the latest history ID seems to guarantee that a subsequent history query will be empty and we can then avoid the junk and start again. Go back to full query if delta fails for any reason. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Fix an exception if message fetching failsJason Gunthorpe2020-02-211-0/+1
| | | | | | | Since size is used to print the log message, even on failure, set it to some value. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Don't leave asyncio tasks running unexpectedlyJason Gunthorpe2020-02-071-2/+3
| | | | | | | | All cases where gather is called intend that the tasks will complete successfully or all cancel at the first error. Add a little wrapper to achieve this. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Allow access to the message headers during mail directingJason Gunthorpe2020-02-071-1/+1
| | | | | | | Access to the headers like List-ID allows some client side filtering of incoming messages. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Do not pass msgdb all over the placeJason Gunthorpe2020-02-071-17/+14
| | | | | | | The mailboxes can only be linked to a single msgdb, always take it from the mailbox cfg. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* Lift Mailbox.cfg out of the sub classesJason Gunthorpe2020-02-071-5/+4
| | | | | | Make sure it is always set Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* GMail supportJason Gunthorpe2020-02-061-0/+639
Basic support for GMail using the REST API Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-11 10:37:55 +0000