rpm/mutt
1
0
Fork 1
forked from pool/mutt
Code Pull Requests Activity

Add oauth2 support

Browse Source 
OBS-URL: https://build.opensuse.org/package/show/server:mail/mutt?expand=0&rev=242
This commit is contained in:
Dr. Werner Fink 2021-07-14 11:16:47 +00:00 committed by Git OBS Bridge
parent 980e09ba67
commit 85c4ff0526
5 changed files with 762 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
backports-datetime-fromisoformat-1.0.0.tar.gz Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9577a2a9486cd7383a5f58b23bb8e81cf0821dbbc0eb7c87d3fa198c1df40f5c
size 10802

7
mutt.changes
View File

@ -1,3 +1,10 @@
-------------------------------------------------------------------
Wed Jul 14 11:07:24 UTC 2021 - Dr. Werner Fink <werner@suse.de>
- Add mutt_oauth2.py ported to python 3.6
- If required the package will include a backport to support
fromisoformat of the python class datetime
-------------------------------------------------------------------
Tue Jul 13 14:37:42 UTC 2021 - Dr. Werner Fink <werner@suse.de>

28
mutt.spec
View File

@ -34,6 +34,9 @@ Source2: README.alternates
Source3: mutt.png
Source4: mutt.desktop
Source5: skel.muttrc
Source6: mutt_oauth2.py-3.6
Source7: mutt_oauth2.py.README
Source8: backports-datetime-fromisoformat-1.0.0.tar.gz
Source9: mutt.mailcap
Patch0: %{name}-1.13.3.dif
# http://www.spinnaker.de/mutt/compressed/
@ -70,6 +73,11 @@ BuildRequires: w3m
BuildRequires: pkgconfig(gssrpc)
BuildRequires: pkgconfig(krb5)
BuildRequires: pkgconfig(libsasl2)
%if 0%{suse_version} >= 1500
BuildRequires: python3-base
BuildRequires: python3-devel
BuildRequires: python3-setuptools
%endif
Requires(post): %{_bindir}/cat
Requires(post): %{_bindir}/mkdir
Requires(postun):%{_bindir}/rm
@ -280,6 +288,18 @@ sed -rn '/Command formats for gpg/,$p' %{SOURCE5} >> %{buildroot}%{_sysconfdir}/
%suse_update_desktop_file mutt
%endif
%if 0%{suse_version} >= 1500
mkdir -p %{buildroot}%{_docdir}/%{name}
install -m 755 %{SOURCE6} %{buildroot}%{_docdir}/%{name}/mutt_oauth2.py
install -m 644 %{SOURCE7} %{buildroot}%{_docdir}/%{name}/mutt_oauth2.py.README
%if %{?pkg_vcmp:%{pkg_vcmp python3-base < 3.7.0}}%{!?pkg_vcmp:0}
tar xf %{SOURCE8}
pushd backports-datetime-fromisoformat-1.0.0
python3 setup.py install --root %{buildroot}
popd
%endif
%endif
%pre
if test $1 -gt 1 -a -e %{_docdir}/%{name}/manual.txt.gz
then
@ -328,6 +348,14 @@ rm -f %{_localstatedir}/adm/update-messages/%{name}-%{version}-%{release}-notify
%{_datadir}/mutt/mailcap
%dir %doc %{_docdir}/%{name}/
%doc %{_docdir}/%{name}/manual.txt.gz
%if 0%{suse_version} >= 1500
%{_docdir}/%{name}/mutt_oauth2.py
%{_docdir}/%{name}/mutt_oauth2.py.README
%if %{?pkg_vcmp:%{pkg_vcmp python3-base < 3.7.0}}%{!?pkg_vcmp:0}
%{python3_sitearch}/backports
%{python3_sitearch}/backports_datetime_fromisoformat-1.0.0-py3.6.egg-info
%endif
%endif
%files doc
%doc %{_docdir}/%{name}/COPYRIGHT

434
mutt_oauth2.py-3.6 Normal file
View File

@ -0,0 +1,434 @@
#!/usr/bin/python3
#
# Mutt OAuth2 token management script, version 2020-08-07
# Written against python 3.7.3, not tried with earlier python versions.
#
# Copyright (C) 2020 Alexander Perlis
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
'''Mutt OAuth2 token management'''
import sys
import json
import argparse
import urllib.parse
import urllib.request
import imaplib
import poplib
import smtplib
import base64
import secrets
import hashlib
import time
if sys.version_info < (3,7):
from datetime import date, timedelta, datetime, time
from backports.datetime_fromisoformat import MonkeyPatch
else:
from datetime import timedelta, datetime
from pathlib import Path
import socket
import http.server
import subprocess
if sys.version_info < (3,7):
from subprocess import PIPE
MonkeyPatch.patch_fromisoformat()
# The token file must be encrypted because it contains multi-use bearer tokens
# whose usage does not require additional verification. Specify whichever
# encryption and decryption pipes you prefer. They should read from standard
# input and write to standard output. The example values here invoke GPG,
# although won't work until an appropriate identity appears in the first line.
ENCRYPTION_PIPE = ['gpg', '--encrypt', '--recipient', 'YOUR_GPG_IDENTITY']
DECRYPTION_PIPE = ['gpg', '--decrypt']
registrations = {
'google': {
'authorize_endpoint': 'https://accounts.google.com/o/oauth2/auth',
'devicecode_endpoint': 'https://oauth2.googleapis.com/device/code',
'token_endpoint': 'https://accounts.google.com/o/oauth2/token',
'redirect_uri': 'urn:ietf:wg:oauth:2.0:oob',
'imap_endpoint': 'imap.gmail.com',
'pop_endpoint': 'pop.gmail.com',
'smtp_endpoint': 'smtp.gmail.com',
'sasl_method': 'OAUTHBEARER',
'scope': 'https://mail.google.com/',
'client_id': '',
'client_secret': '',
},
'microsoft': {
'authorize_endpoint': 'https://login.microsoftonline.com/common/oauth2/v2.0/authorize',
'devicecode_endpoint': 'https://login.microsoftonline.com/common/oauth2/v2.0/devicecode',
'token_endpoint': 'https://login.microsoftonline.com/common/oauth2/v2.0/token',
'redirect_uri': 'https://login.microsoftonline.com/common/oauth2/nativeclient',
'tenant': 'common',
'imap_endpoint': 'outlook.office365.com',
'pop_endpoint': 'outlook.office365.com',
'smtp_endpoint': 'smtp.office365.com',
'sasl_method': 'XOAUTH2',
'scope': ('offline_access https://outlook.office.com/IMAP.AccessAsUser.All '
'https://outlook.office.com/POP.AccessAsUser.All '
'https://outlook.office.com/SMTP.Send'),
'client_id': '',
'client_secret': '',
},
}
ap = argparse.ArgumentParser(epilog='''
This script obtains and prints a valid OAuth2 access token. State is maintained in an
encrypted TOKENFILE. Run with "--verbose --authorize" to get started or whenever all
tokens have expired, optionally with "--authflow" to override the default authorization
flow. To truly start over from scratch, first delete TOKENFILE. Use "--verbose --test"
to test the IMAP/POP/SMTP endpoints.
''')
ap.add_argument('-v', '--verbose', action='store_true', help='increase verbosity')
ap.add_argument('-d', '--debug', action='store_true', help='enable debug output')
ap.add_argument('tokenfile', help='persistent token storage')
ap.add_argument('-a', '--authorize', action='store_true', help='manually authorize new tokens')
ap.add_argument('--authflow', help='authcode | localhostauthcode | devicecode')
ap.add_argument('-t', '--test', action='store_true', help='test IMAP/POP/SMTP endpoints')
args = ap.parse_args()
token = {}
path = Path(args.tokenfile)
if path.exists():
if 0o777 & path.stat().st_mode != 0o600:
sys.exit('Token file has unsafe mode. Suggest deleting and starting over.')
try:
if sys.version_info < (3,7):
sub = subprocess.run(DECRYPTION_PIPE, check=True, input=path.read_bytes(),
stdout=PIPE, stderr=PIPE)
else:
sub = subprocess.run(DECRYPTION_PIPE, check=True, input=path.read_bytes(),
capture_output=True)
token = json.loads(sub.stdout)
except subprocess.CalledProcessError:
sys.exit('Difficulty decrypting token file. Is your decryption agent primed for '
'non-interactive usage, or an appropriate environment variable such as '
'GPG_TTY set to allow interactive agent usage from inside a pipe?')
def writetokenfile():
'''Writes global token dictionary into token file.'''
if not path.exists():
path.touch(mode=0o600)
if 0o777 & path.stat().st_mode != 0o600:
sys.exit('Token file has unsafe mode. Suggest deleting and starting over.')
if sys.version_info < (3,7):
sub2 = subprocess.run(ENCRYPTION_PIPE, check=True, input=json.dumps(token).encode(),
stdout=PIPE, stderr=PIPE)
else:
sub2 = subprocess.run(ENCRYPTION_PIPE, check=True, input=json.dumps(token).encode(),
capture_output=True)
path.write_bytes(sub2.stdout)
if args.debug:
print('Obtained from token file:', json.dumps(token))
if not token:
if not args.authorize:
sys.exit('You must run script with "--authorize" at least once.')
print('Available app and endpoint registrations:', *registrations)
token['registration'] = input('OAuth2 registration: ')
token['authflow'] = input('Preferred OAuth2 flow ("authcode" or "localhostauthcode" '
'or "devicecode"): ')
token['email'] = input('Account e-mail address: ')
token['access_token'] = ''
token['access_token_expiration'] = ''
token['refresh_token'] = ''
writetokenfile()
if token['registration'] not in registrations:
sys.exit(f'ERROR: Unknown registration "{token["registration"]}". Delete token file '
f'and start over.')
registration = registrations[token['registration']]
authflow = token['authflow']
if args.authflow:
authflow = args.authflow
baseparams = {'client_id': registration['client_id']}
# Microsoft uses 'tenant' but Google does not
if 'tenant' in registration:
baseparams['tenant'] = registration['tenant']
def access_token_valid():
'''Returns True when stored access token exists and is still valid at this time.'''
token_exp = token['access_token_expiration']
return token_exp and datetime.now() < datetime.fromisoformat(token_exp)
def update_tokens(r):
'''Takes a response dictionary, extracts tokens out of it, and updates token file.'''
token['access_token'] = r['access_token']
token['access_token_expiration'] = (datetime.now() +
timedelta(seconds=int(r['expires_in']))).isoformat()
if 'refresh_token' in r:
token['refresh_token'] = r['refresh_token']
writetokenfile()
if args.verbose:
print(f'NOTICE: Obtained new access token, expires {token["access_token_expiration"]}.')
if args.authorize:
p = baseparams.copy()
p['scope'] = registration['scope']
if authflow in ('authcode', 'localhostauthcode'):
verifier = secrets.token_urlsafe(90)
challenge = base64.urlsafe_b64encode(hashlib.sha256(verifier.encode()).digest())[:-1]
redirect_uri = registration['redirect_uri']
listen_port = 0
if authflow == 'localhostauthcode':
# Find an available port to listen on
s = socket.socket()
s.bind(('127.0.0.1', 0))
listen_port = s.getsockname()[1]
s.close()
redirect_uri = 'http://localhost:'+str(listen_port)+'/'
# Probably should edit the port number into the actual redirect URL.
p.update({'login_hint': token['email'],
'response_type': 'code',
'redirect_uri': redirect_uri,
'code_challenge': challenge,
'code_challenge_method': 'S256'})
print(registration["authorize_endpoint"] + '?' +
urllib.parse.urlencode(p, quote_via=urllib.parse.quote))
authcode = ''
if authflow == 'authcode':
authcode = input('Visit displayed URL to retrieve authorization code. Enter '
'code from server (might be in browser address bar): ')
else:
print('Visit displayed URL to authorize this application. Waiting...',
end='', flush=True)
class MyHandler(http.server.BaseHTTPRequestHandler):
'''Handles the browser query resulting from redirect to redirect_uri.'''
# pylint: disable=C0103
def do_HEAD(self):
'''Response to a HEAD requests.'''
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
def do_GET(self):
'''For GET request, extract code parameter from URL.'''
# pylint: disable=W0603
global authcode
querystring = urllib.parse.urlparse(self.path).query
querydict = urllib.parse.parse_qs(querystring)
if 'code' in querydict:
authcode = querydict['code'][0]
self.do_HEAD()
self.wfile.write(b'<html><head><title>Authorizaton result</title></head>')
self.wfile.write(b'<body><p>Authorization redirect completed. You may '
b'close this window.</p></body></html>')
with http.server.HTTPServer(('127.0.0.1', listen_port), MyHandler) as httpd:
try:
httpd.handle_request()
except KeyboardInterrupt:
pass
if not authcode:
sys.exit('Did not obtain an authcode.')
for k in 'response_type', 'login_hint', 'code_challenge', 'code_challenge_method':
del p[k]
p.update({'grant_type': 'authorization_code',
'code': authcode,
'client_secret': registration['client_secret'],
'code_verifier': verifier})
try:
response = urllib.request.urlopen(registration['token_endpoint'],
urllib.parse.urlencode(p).encode())
except urllib.error.HTTPError as err:
print(err.code, err.reason)
response = err
response = response.read()
if args.debug:
print(response)
response = json.loads(response)
if 'error' in response:
print(response['error'])
if 'error_description' in response:
print(response['error_description'])
sys.exit(1)
elif authflow == 'devicecode':
try:
response = urllib.request.urlopen(registration['devicecode_endpoint'],
urllib.parse.urlencode(p).encode())
except urllib.error.HTTPError as err:
print(err.code, err.reason)
response = err
response = response.read()
if args.debug:
print(response)
response = json.loads(response)
if 'error' in response:
print(response['error'])
if 'error_description' in response:
print(response['error_description'])
sys.exit(1)
print(response['message'])
del p['scope']
p.update({'grant_type': 'urn:ietf:params:oauth:grant-type:device_code',
'client_secret': registration['client_secret'],
'device_code': response['device_code']})
interval = int(response['interval'])
print('Polling...', end='', flush=True)
while True:
time.sleep(interval)
print('.', end='', flush=True)
try:
response = urllib.request.urlopen(registration['token_endpoint'],
urllib.parse.urlencode(p).encode())
except urllib.error.HTTPError as err:
# Not actually always an error, might just mean "keep trying..."
response = err
response = response.read()
if args.debug:
print(response)
response = json.loads(response)
if 'error' not in response:
break
if response['error'] == 'authorization_declined':
print(' user declined authorization.')
sys.exit(1)
if response['error'] == 'expired_token':
print(' too much time has elapsed.')
sys.exit(1)
if response['error'] != 'authorization_pending':
print(response['error'])
if 'error_description' in response:
print(response['error_description'])
sys.exit(1)
print()
else:
sys.exit(f'ERROR: Unknown OAuth2 flow "{token["authflow"]}. Delete token file and '
f'start over.')
update_tokens(response)
if not access_token_valid():
if args.verbose:
print('NOTICE: Invalid or expired access token; using refresh token '
'to obtain new access token.')
if not token['refresh_token']:
sys.exit('ERROR: No refresh token. Run script with "--authorize".')
p = baseparams.copy()
p.update({'client_secret': registration['client_secret'],
'refresh_token': token['refresh_token'],
'grant_type': 'refresh_token'})
try:
response = urllib.request.urlopen(registration['token_endpoint'],
urllib.parse.urlencode(p).encode())
except urllib.error.HTTPError as err:
print(err.code, err.reason)
response = err
response = response.read()
if args.debug:
print(response)
response = json.loads(response)
if 'error' in response:
print(response['error'])
if 'error_description' in response:
print(response['error_description'])
print('Perhaps refresh token invalid. Try running once with "--authorize"')
sys.exit(1)
update_tokens(response)
if not access_token_valid():
sys.exit('ERROR: No valid access token. This should not be able to happen.')
if args.verbose:
print('Access Token: ', end='')
print(token['access_token'])
def build_sasl_string(user, host, port, bearer_token):
'''Build appropriate SASL string, which depends on cloud server's supported SASL method.'''
if registration['sasl_method'] == 'OAUTHBEARER':
return f'n,a={user},\1host={host}\1port={port}\1auth=Bearer {bearer_token}\1\1'
if registration['sasl_method'] == 'XOAUTH2':
return f'user={user}\1auth=Bearer {bearer_token}\1\1'
sys.exit(f'Unknown SASL method {registration["sasl_method"]}.')
if args.test:
errors = False
imap_conn = imaplib.IMAP4_SSL(registration['imap_endpoint'])
sasl_string = build_sasl_string(token['email'], registration['imap_endpoint'], 993,
token['access_token'])
if args.debug:
imap_conn.debug = 4
try:
imap_conn.authenticate(registration['sasl_method'], lambda _: sasl_string.encode())
# Microsoft has a bug wherein a mismatch between username and token can still report a
# successful login... (Try a consumer login with the token from a work/school account.)
# Fortunately subsequent commands fail with an error. Thus we follow AUTH with another
# IMAP command before reporting success.
imap_conn.list()
if args.verbose:
print('IMAP authentication succeeded')
except imaplib.IMAP4.error as e:
print('IMAP authentication FAILED (does your account allow IMAP?):', e)
errors = True
pop_conn = poplib.POP3_SSL(registration['pop_endpoint'])
sasl_string = build_sasl_string(token['email'], registration['pop_endpoint'], 995,
token['access_token'])
if args.debug:
pop_conn.set_debuglevel(2)
try:
# poplib doesn't have an auth command taking an authenticator object
# Microsoft requires a two-line SASL for POP
# pylint: disable=W0212
pop_conn._shortcmd('AUTH ' + registration['sasl_method'])
pop_conn._shortcmd(base64.standard_b64encode(sasl_string.encode()).decode())
if args.verbose:
print('POP authentication succeeded')
except poplib.error_proto as e:
print('POP authentication FAILED (does your account allow POP?):', e.args[0].decode())
errors = True
# SMTP_SSL would be simpler but Microsoft does not answer on port 465.
smtp_conn = smtplib.SMTP(registration['smtp_endpoint'], 587)
sasl_string = build_sasl_string(token['email'], registration['smtp_endpoint'], 587,
token['access_token'])
smtp_conn.ehlo('test')
smtp_conn.starttls()
smtp_conn.ehlo('test')
if args.debug:
smtp_conn.set_debuglevel(2)
try:
smtp_conn.auth(registration['sasl_method'], lambda _=None: sasl_string)
if args.verbose:
print('SMTP authentication succeeded')
except smtplib.SMTPAuthenticationError as e:
print('SMTP authentication FAILED:', e)
errors = True
if errors:
sys.exit(1)

290
mutt_oauth2.py.README Normal file
View File

@ -0,0 +1,290 @@
mutt_oauth.py README by Alexander Perlis, 2020-07-15
====================================================
Background on plain passwords, app passwords, OAuth2 bearer tokens
------------------------------------------------------------------
An auth stage occurs near the start of the IMAP/POP/SMTP protocol
conversation. Various SASL methods can be used (depends on what the
server offers, and what the client supports). The PLAIN method, also
known as "basic auth", involves simply sending the username and
password (this occurs over an encrypted connection), and used to be
common; but, for large cloud mail providers, basic auth is a security
hole. User passwords often have low entropy (humans generally choose
passwords that can be produced from human memory), thus are targets
for various types of exhaustive attacks. Older attacks try different
passwords against one user, whereas newer spray attacks try one
password against different users. General mitigation efforts such as
rate-limiting, or detection and outright blocking efforts, lead to
degraded or outright denied services for legitimate users. The
security weakness is two-fold: the low entropy of the user password,
together with the alarming consequence that the password often unlocks
many disparate systems in a typical enterprise single-sign-on
environment. Also, humans type passwords or copy/paste them from
elsewhere on the screen, so they can also be grabbed via keyloggers or
screen capture (or a human bystander). Two ways to solve these
conundrums:
- app passwords
- bearer tokens
App passwords are simply high-entropy protocol-specific passwords, in
other words a long computer-generated random string, you use one for
your mail system, a different one for your payroll system, and so
on. With app passwords in use, brute-force attacks become useless. App
passwords require no modifications to client software, and only minor
changes on the server side. One way to think about app passwords is
that they essentially impose on you the use of a password manager. Any
user can go to the trouble of using a password manager but most users
don't bother. App passwords put the password manager inside the server
and force you to use it.
Bearer tokens take the idea of app passwords to the next level. Much
like app passwords, they too are just long computer-generated random
strings, knowledge of which simply "lets you in". But unlike an app
password which the user must manually copy from a server password
screen and then paste into their client account config screen (a
process the user doesn't want to follow too often), bearer tokens get
swapped out approximately once an hour without user interaction. For
this to work, both clients and servers must be modified to speak a
separate out-of-band protocol (the "OAuth2" protocol) to swap out
tokens. More precisely, from start to finish, the process goes like
this: the client and server must once-and-for-all be informed about
each other (this is called "app registration" and might be done by the
client developer or left to each end user), then the client informs
the server that it wants to connect, then the user is informed to
independently use a web browser to visit a server destination to
approve this request (at this stage the server will require the user
to authenticate using say their password and perhaps additional
factors such as an SMS verification or crypto device), then the client
will have a long-term "refresh token" as well as an "access token"
good for about an hour. The access token can now be used with
IMAP/POP/SMTP to access the account. When it expires, the refresh
token is used to get a new access token and perhaps a new refresh
token. After several months of such usage, even the refresh token may
expire and the human user will have to go back and re-authenticate
(password, SMS, crypto device, etc) for things to start anew.
Since app passwords and tokens are high-entropy and their compromise
should compromise only a particular system (rather than all systems in
a single-sign-on environment), they have similar security strength
when compared to stark weakness of traditional human passwords. But if
compared only to each other, tokens provide more security. App
passwords must be short enough for humans to easily copy/paste them,
might get written down or snooped during that process, and anyhow are
long-lived and thus could get compromised by other means. The main
drawback to tokens is that their support requires significant changes
to clients and servers, but once such support exists, they are
superior and easier to use.
Many cloud providers are eliminating support for human passwords. Some are
allowing app passwords in addition to tokens. Some allow only tokens.
OAuth2 token support in mutt
----------------------------
Mutt supports the two SASL methods OAUTHBEARER and XOAUTH2 for presenting an
OAuth2 access token near the start of the IMAP/POP/SMTP connection.
(Two different SASL methods exist for historical reasons. While OAuth2
was under development, the experimental offering by servers was called
XOAUTH2, later fleshed out into a standard named OAUTHBEARER, but not
all servers have been updated to offer OAUTHBEARER. Once the major
cloud providers all support OAUTHBEARER, clients like mutt might be
modified to no longer know about XOAUTH2.)
Mutt can present a token inside IMAP/POP/SMTP, but by design mutt itself
does not know how to have a separate conversation (outside of IMAP/POP/SMTP)
with the server to authorize the user and obtain refresh and access tokens.
Mutt just needs an access token, and has a hook for an external script to
somehow obtain one.
mutt_oauth2.py is an example of such an external script. It likely can be
adapted to work with OAuth2 on many different cloud mail providers, and has
been tested against:
- Google consumer account (@gmail.com)
- Google work/school account (G Suite tenant)
- Microsoft consumer account (e.g., @live.com, @outlook.com, ...)
- Microsoft work/school account (Azure tenant)
(Note that Microsoft uses the marketing term "Modern Auth" in lieu of
"OAuth2". In that terminology, mutt indeed supports "Modern Auth".)
Configure script's token file encryption
----------------------------------------
The script remembers tokens between invocations by keeping them in a
token file. This file is encrypted. Inside the script are two lines
ENCRYPTION_PIPE
DECRYPTION_PIPE
that must be edited to specify your choice of encryption system. A
popular choice is gpg. To use this:
- Install gpg. For example, "sudo apt install gpg".
- "gpg --gen-key". Answer the questions. Instead of your email
address you could choose say "My mutt_oauth2 token store", then
choose a passphrase. You will need to produce that same passphrase
whenever mutt_oauth2 needs to unlock the token store.
- Edit mutt_oauth2.py and put your GPG identity (your email address or
whatever you picked above) in the ENCRYPTION_PIPE line.
- For the gpg-agent to be able to ask you the unlock passphrase,
the environment variable GPG_TTY must be set to the current tty.
Typically you would put the following inside your .bashrc or equivalent:
export GPG_TTY=$(tty)
Create an app registration
--------------------------
Before you can connect the script to an account, you need an
"app registration" for that service. Cloud entities (like Google and
Microsoft) and/or the tenant admins (the central technology admins at
your school or place of work) might be restrictive in who can create
app registrations, as well as who can subsequently use them. For
personal/consumer accounts, you can generally create your own
registration and then use it with a limited number of different personal
accounts. But for work/school accounts, the tenant admins might approve an
app registration that you created with a personal/consumer account, or
might want an official app registration from a developer (the creation of
which and blessing by the cloud provider might require payment and/or arduous
review), or might perhaps be willing to roll their own "in-house" registration.
What you ultimately need is the "client_id" (and "client_secret" if
one was set) for this registration. Those values must be edited into
the mutt_oauth2.py script. If your work or school environment has a
knowledge base that provides the client_id, then someone already took
care of the app registration, and you can skip the step of creating
your own registration.
-- How to create a Google registration --
Go to console.developers.google.com, and create a new project. The name doesn't
matter and could be "mutt registration project".
- Go to Library, choose Gmail API, and enable it
- Hit left arrow icon to get back to console.developers.google.com
- Choose OAuth Consent Screen
- Choose Internal for an organizational G Suite
- Choose External if that's your only choice
- For Application Name, put for example "Mutt"
- Under scopes, choose Add scope, scroll all the way down, enable the "https://mail.google.com/" scope
- Fill out additional fields (application logo, etc) if you feel like it (will make the consent screen look nicer)
- Back at console.developers.google.com, choose Credentials
- At top, choose Create Credentials / OAuth2 client iD
- Application type is "Desktop app"
Edit the client_id (and client_secret if there is one) into the
mutt_oauth2.py script.
-- How to create a Microsoft registration --
Go to portal.azure.com, log in with a Microsoft account (get a free
one at outlook.com), then search for "app registration", and add a
new registration. On the initial form that appears, put a name like
"Mutt", allow any type of account, and put "http://localhost/" as
the redirect URI, then more carefully go through each
screen:
Branding
- Leave fields blank or put in reasonable values
- For official registration, verify your choice of publisher domain
Authentication:
- Platform "Mobile and desktop"
- Redirect URI "http://localhost/"
- Any kind of account
- Enable public client (allow device code flow)
API permissions:
- Microsoft Graph, Delegated, "offline_access"
- Microsoft Graph, Delegated, "IMAP.AccessAsUser.All"
- Microsoft Graph, Delegated, "POP.AccessAsUser.All"
- Microsoft Graph, Delegated, "SMTP.Send"
- Microsoft Graph, Delegated, "User.Read"
Overview:
- Take note of the Application ID (a.k.a. Client ID), you'll need it shortly
End users who aren't able to get to the app registration screen within
portal.azure.com for their work/school account can temporarily use an
incognito browser window to create a free outlook.com account and use that
to create the app registration.
Edit the client_id (and client_secret if there is one) into the
mutt_oauth2.py script.
Running the script manually to authorize tokens
-----------------------------------------------
Run "mutt_oauth2.py --help" to learn script usage. To obtain the
initial set of tokens, run the script specifying a name for a
disposable token storage file, as well as "--authorize", for example
using this naming scheme:
mutt_oauth2.py userid@myschool.edu.tokens --verbose --authorize
The script will ask questions and provide some instructions. For the
flow question:
- "authcode": you paste a complicated URL into a browser, then
manually extract a "code" parameter from a subsequent URL in the
browser address bar and paste that back to the script.
- "localhostauthcode": you again paste the complicated URL into a browser
but that's it --- the code is automatically extracted from the response
relying on a localhost redirect and temporarily listening on a localhost
port. This flow can only be used if the web browser opening the redirect
URL sits on the same machine as where mutt is running, in other words can not
be used if you ssh to a remote machine and run mutt on that remote machine
while your web browser remains on your local machine.
- "devicecode": you go to a simple URL and just enter a short code.
Your answer here determines the default flow, but on any invocation of
the script you can override the default with the optional "--authflow"
parameter. To change the default, delete your token file and start over.
To figure out which flow to use, I suggest trying all three.
Depending on the OAuth2 provider and how the app registration was
configured, some flows might not work, so simply trying them is the
best way to figure out what works and which one you prefer. Personally
I prefer the "localhostauthcode" flow when I can use it.
Once you attempt an actual authorization, you might get stuck because
the web browser step might indicate your institution admins must grant
approval. Indeed engage them in a conversation about approving the
use of mutt to access mail. If that fails, an alternative is to
identify some other well-known IMAP/POP/SMTP client that they might
have already approved, or might be willing to approve, and first go
configure it for OAuth2 and see whether it will work to reach your
mail, and then you could dig into the source code for that client and
extract its client_id, client_secret, and redirect_uri and put those
into the mutt_oauth2.py script. This would be a temporary punt for
end-user experimentation, but not an approach for configuring systems
to be used by other people. Engaging your institution admins to create
a mutt registration is the better way to go.
Once you've succeeded authorizing mutt_oauth2.py to obtain tokens, try
one of the following to see whether IMAP/POP/SMTP are working:
mutt_oauth2.py userid@myschool.edu.tokens --verbose --test
mutt_oauth2.py userid@myschool.edu.tokens --verbose --debug --test
Without optional parameters, the script simply returns an access token
(possibly first conducting a behind-the-scenes URL retrieval using a
stored refresh token to obtain an updated access token). Calling the
script without optional parameters is how it will be used by
mutt. Your .muttrc would look something like:
set imap_user="userid@myschool.edu"
set folder="imap://outlook.office365.com/"
set smtp_url="smtp://${imap_user}@smtp.office365.com:587/"
set imap_authenticators="oauthbearer:xoauth2"
set imap_oauth_refresh_command="/path/to/script/mutt_oauth2.py ${imap_user}.tokens"
set smtp_authenticators=${imap_authenticators}
set smtp_oauth_refresh_command=${imap_oauth_refresh_command}