certbot has now been updated so you should be able to get Let's Encrypt certificates again.It looks like you need to have a default Web Server configured through the webconfig in order for the Let's Encrypt app to work.

Hi Nuke,
If you install the Let's Encrypt app, it will take over your current certificate and continue to maintain it so you can still use it. There should be no problem installing the app from that perspective. The only issue may be unbolting the changes you made to get certbot installed in the first place. For me this meant just removing my cron job so the ClearOS job could look after the renewal. My Apache was already configured to use the certificates in /etc/letsencrypt/live/{my_domain} so I did not need any changes there and I had not set up Webconfig to use them (nor mail).

There does appear to be a bug in the app. It looks like it should be able to create a certificate for multiple domains in one go but it fails with an error.

[edit]
It looks like it is not a ClearOS bug. A vulnerability was discovered in the protocol used to do multiple certificated so Let's Encrypt shut it down for the moment. A new version of certbot is required with an alternative method of authentication and it is in testing at the moment at Let's Encrypt. It then needs to filter through to ClearOS.
[/edit]

Nick Howitt wrote:
I kept my options open when creating the initial certificate with certbot before the Let's Encrypt app was released. With the Let's Encrypt App, it can only cover one FQDN. I believe that from 27th February, Let's Encrypt will release a wildcard certificate which makes life easier but I have no idea if you can request one with the app.

Nick, I don't think I understand what you mean.
When I did the original install I created 1 certificate for my home domain and also one virtual domain within the same certificate as you have. So with the MarketPlace App you can't do this? Since i need it to cover 2 domains, I should stay with my present manual installation and not change to the Marketplace App?
Thanks again for your help!

I've done some more testing. The FQDN of your mail server that you use in your client *must* match a name covered by your certificate or you get a warning/error in your mail client. In my case I have a basic certificate for www.howitts.co.uk but, from the "X509v3 Subject Alternative Name", it covers:

howitts.co.uk

howitts.poweredbyclear.com

lanserver.howitts.co.uk

mailserver.howitts.co.uk

server.howitts.co.uk

www.howitts.co.uk

I kept my options open when creating the initial certificate with certbot before the Let's Encrypt app was released. With the Let's Encrypt App, it can only cover one FQDN. I believe that from 27th February, Let's Encrypt will release a wildcard certificate which makes life easier but I have no idea if you can request one with the app.

Derek Hirst wrote:
# Encryption with TLS
smtpd_tls_auth_only = yes
smtpd_use_tls = yes
smtpd_tls_received_header = yes
smtpd_tls_cert_file = /etc/letsencrypt/live/nutterpc.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/nutterpc.com/privkey.pem
smtpd_tls_loglevel = 1
smtp_tls_security_level = may
smtp_tls_CAfile = /etc/letsencrypt/live/nutterpc.com/isrgrootx1.pem

Be careful here. You are mixing and matching smtp and smtpd parameters, One set is for the server, the other the client.

In your imapd.conf you also have an error. If using Let's Encrypt, leave the CA file as:

tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt

Also try to avoid copying the certificates around. Remember they expire every three months. It is best to point directly to the files in /etc/letsencrypt/live and change the folder ownership to root:mail and permissions to 0750. You then need to restart cyrus-imapd/postfix every time the certificates are renewed. Details are in the thread (although untested)

Re domain records. It makes sense that your mx record and certificates match their names and the server name used in your mail apps. Mine do as I originally created a Let's Encrypt certificate for multiple domains including mailserver.howitts.co.uk which is where my mx record points to. On my LAN it resolves to my ClearOS server as well. With the Let's Encrypt app you an create another certificate for you mail server, or even two, one for imap.your_domain.com and smtp.your_domain.com and so on or just use a www certificate and then use that name as the mail server in your clients. That is up to you with how you want to arrange your domain names.

I've given all the edits you need to make in the thread. Perhaps I need to pull together a HowTo, but I still need to test what happens if your client does not use an FQDN covered by your certificate. Time is a bit pressured for the next couple of weeks.

I've also just found the error that you were experiencing, didn't take long

I'm working on finding an easy way to resolve it, hold tight

SOLUTION:

I didn't think this was too big of an issue, but Nick's post gave me an idea as to what needed to happen (Permissions based)

All I've done, is using WinSCP, is login to the server via SSH, gone to the letsencrypt folder where the certificates are stored. Grab them all, copy them across to your PC

Then change directory to /etc/pki/cryus-imapd/

Go back to your computer, grab the pem files you just downloaded earlier, and upload them into the afore mentioned folder. Ensure you have told imapd.conf about this:

tls_cert_file: /etc/pki/cyrus-imapd/fullchain.pem
tls_key_file: /etc/pki/cyrus-imapd/privkey.pem
tls_ca_file: /etc/pki/cyrus-imapd/isrgrootx1.pem

Save and exit. restart both cyrus-imapd & postfix, have a terminal running with the following command:

tail -f /var/log/maillog

This way you can watch the mail server. And also keep an eye out for dialog boxes which will come up (one did for me) as it will prompt you about this new certificate it has encountered (which is the one we just moved to its new home) accept that and then voila!

It's all done my friend

Ah reading the errors, you've come into the same issues i did when I first got myself going:

Regarding the following error:

Jan 9 10:23:49 mydomain imaps[1237]: Fatal error: tls_init() failed
Jan 9 10:23:49 mydomain imaps[1243]: TLS server engine: cannot load CA data
Jan 9 10:23:49 mydomain imaps[1243]: unable to get certificate from '/etc/pki/tls/certs/mail.mydomain.com.crt.pem'

It is indeed possible to get it running with letsencrypt. I'll post a link as to how I've since gotten mine running. You will also need to ensure you have the correct domain records set if you go to add a SPF/DKIM record (this can be a touchy area)

This is how mine looks:

# Authentication with SASL
broken_sasl_auth_clients = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $mydomain
smtpd_sasl_type = cyrus

# Encryption with TLS
smtpd_tls_auth_only = yes
smtpd_use_tls = yes
smtpd_tls_received_header = yes
smtpd_tls_cert_file = /etc/letsencrypt/live/nutterpc.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/nutterpc.com/privkey.pem
smtpd_tls_loglevel = 1
smtp_tls_security_level = may
smtp_tls_CAfile = /etc/letsencrypt/live/nutterpc.com/isrgrootx1.pem

The error you are experiencing with regards to the "unable to get certificate" is regarding the certificate file being used, it just doesn't contain all the necessary data required for postfix to be able to use/load it

What happens with the settings i have above, is STARTTLS initiated connection, then full encryption during connection to the server for sending of emails (I don't have a sample to be able to show as my emails arent coming in atm, no one loves me, hahaha

If you do still need help, gimme a yell. I didn't find it too bad getting mine running. But now that it IS running, its lovely knowing that you have complete control over your emails

My K-9 has SSL/TLS for IMAPS and it works without errors. Oddly, the maillog shows it negotiating a STARTTLS connection with TLSv1.2. If you select STARTTLS, K-9 switches back to port 143. If it was a foreign IP perhaps it cyrus-imap bombed bombed because authentication failed or the person did not even attempt to authenticate.

Nick Howitt wrote:
I don't know about your last error. Are you connecting to cyrus-imap using STARTTLS (SSL/TLS) or SSL?

I believe the options in Thunderbird & K-9 are STARTTLS or SSL/TLS. I have SSL/TLS enabled for imaps and pops email. Regular imap and pop are disabled.

The error happen right after an attempted login failed. I think the failed login is good because I don't know the IP address but there shouldn't be a fatal error??

imaps[24676]: imaps TLS negotiation failed: [IP removed]

imaps[24676]: Fatal error: tls_start_servertls() failed]

[edit]
added more info about error
[/edit]

The Let's Encrypt app uses certbot underneath and will just take over any pre-existing Let's Encrypt certificates and its entire directory structure. This means you can use your current ones - it will save you acknowledging them in your clients. It is what I did. I don't know which method you used to implement certificate renewal with certbot. I just used a one line cron-daily entry and that was easy to add restarting of postfix and cyrus-imapd to the end of it. The ClearOS app will provide its own renewal mechanism and I have referenced that earlier in the thread.

I have implemented DKIM and have some notes on it. I also use SPF.

I would expect key permissions to be the same as the certificate ones. If you look at the original cyrus-imapd certificate, you should find it is a single file containing a certificate and key. Generally apps requiring certificates can handle the single file format containing any or all of the CA, certificate, chain/intermediate and key as I think it is a feature of the OpenSSL libraries.

I don't know about your last error. Are you connecting to cyrus-imap using STARTTLS (SSL/TLS) or SSL?

Nick Howitt wrote:
I've changed ownership of /etc/letsencrypt/live/ to root:mail and permissions to 0750. On /etc/letsencrypt/live/{my_domain.com}/, permissions are 0755 and ownership is root:root. The symlinks below are 0777 and root:root. It is the same for /etc/letsencrypt/archive/ and /etc/letsencrypt/archive/{my_domain.com}/. I do not know what the original ones were. It looks like I made the change in July and tried changing the actual certificates to root:ssl-cert, but the later ones which certbot got when renewing are root:root. You may not need to change ownership of /etc/letsencrypt/live/ if you change permissions to 755.
...
The default certificate, /etc/pki/cyrus-imapd/cyrus-imapd.pem, is 0644, so world readable.

Thanks Nick!

I did some more changes and kaboom!! it is working. What I did in case someone reads this (or if I forget ...)

My cyrus-imapd was different than yours.

-rw-r----- 1 root mail 3242 Sep  8 11:44 cyrus-imapd.pem

So what I did is a bit different. I want to use my self generated certificate because my Let's Encrypt needs to be moved to the MarketApp when I have time - I have some questions that I'll add onto the earlier post. Then I have to investigate this DKIM etc.

I changed group to mail & permission 0640 on my crt & key files that were converted to pem. I figured this to be the same as it was in my install. Not quite sure that is 100% for the key file??

cyrus-imapd started without any errors. So I think I've solved that part with your help. Thanks so much!

One more question.
When I see

imaps[5810]: imaps TLS negotiation failed: dIP-of-Domain.domain.com [IP address]

imaps[5810]: Fatal error: tls_start_servertls() failed

this, is this a problem with the domain.com not using TLS1.2? I think I know who this is and confirming.
What do you do about this?

Hi Nuke, bedtime beckoned after my last post!

I've changed ownership of /etc/letsencrypt/live/ to root:mail and permissions to 0750. On /etc/letsencrypt/live/{my_domain.com}/, permissions are 0755 and ownership is root:root. The symlinks below are 0777 and root:root. It is the same for /etc/letsencrypt/archive/ and /etc/letsencrypt/archive/{my_domain.com}/. I do not know what the original ones were. It looks like I made the change in July and tried changing the actual certificates to root:ssl-cert, but the later ones which certbot got when renewing are root:root. You may not need to change ownership of /etc/letsencrypt/live/ if you change permissions to 755.

This has no effect on apache at all which continues to use the certificate

I'll add that my single certificate covers mailserver.howitts.co.uk, www.howitts.co.uk and a few other names plus my poweredbyclear.com FQDN.

Also, it implies that all your other issues were probably permission related. The default certificate, /etc/pki/cyrus-imapd/cyrus-imapd.pem, is 0644, so world readable.

[edit]
Typos and rubbish forum formatting fixed.
[/edit]

Thanks Nick, I just found that link a moment ago too.

My imapd.conf file looks nearly identical. I don't have the bottom expire days etc.

Right now
/etc/letsencrypt/live is 0700 root root
/etc/letsencrypt/live/mydomain.com is 0755 root root
the files in /etc/letsencrypt/live/mydomain.com/ are all linked to ../../archive/mydomain.com/{cert3.pem|chain3.pem|fullchain3.pem|privkey3.pem} and are 777 (lrwxrwxrwx) root root

So if I understand this correctly, you changed the owner of the /etc/letsencrypt/live/ folder to "mail" and 0750.

Doesn't this mess up the access for the certificates for apache?

link

I agree with you - perhaps something like an OpenSSL problem. I may have to try some research tomorrow. My imapd.conf is pretty standard:

configdirectory: /var/lib/imap

partition-default: /var/spool/imap

admins: root

sievedir: /var/lib/imap/sieve

sendmail: /usr/sbin/sendmail

hashimapspool: true

sasl_pwcheck_method: saslauthd

sasl_mech_list: PLAIN

#tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem

#tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem

tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt



# changes by njh

# I also had to change /etc/letsencrypt/live and possibly /etc/letsencrypt/archive group ownership to mail and permissions to 750

tls_key_file:    /etc/letsencrypt/live/www.howitts.co.uk/privkey.pem

tls_cert_file:   /etc/letsencrypt/live/www.howitts.co.uk/fullchain.pem

tls_ca_path:	/etc/pki/tls



flushseenstate:       1

allowplaintext:       yes

reject8bit:           no

munge8bit:            no

lmtp_over_quota_perm_failure: 1

timeout:              30

imapidlepoll:         60

idlesocket:           /var/lib/imap/socket/idle

lmtpsocket:           /var/lib/imap/socket/lmtp

allowapop:            no

altnamespace:         0

unixhierarchysep:     yes

lmtp_downcase_rcpt:   yes

username_tolower:     1

autocreatequota:      -1

createonpost:         1

virtdomains:          off

expire-days:	1

expunge-days:	1

delete-days:	7

The lines at the end are added my me. Note the comment I made further up. It is old so I can't remember if it is necessary or if there is a certificate group or something like that which sorts it all out.

Aaaaaarrrrrggggghhhh!

I tried the Let's Encrypt certificates but get the same error. I suspect there is something else at play. I have a plain default install of Cyrus (marketplace) and postfix.

I only started changing the config files over the weekend as I finally had time to create a better certificate for email rather than the localhost.localdomain. There is no problem with postfix using the self generated .crt and .key files as far as I can tell from the logs. I'm getting the same error using the Let's Encrypt cert.pem.

Jan  9 16:12:58 mydomain imaps[10541]: unable to get certificate from '/etc/letsencrypt/live/mydomain.com/cert.pem'

Jan  9 16:12:58 mydomain imaps[10541]: TLS server engine: cannot load cert/key data

Jan  9 16:12:58 mydomain imaps[10541]: error initializing TLS

Jan  9 16:12:58 mydomain imaps[10541]: Fatal error: tls_init() failed

Jan  9 16:15:46 mydomain pop3s[11233]: unable to get certificate from '/etc/letsencrypt/live/mydomain.com/cert.pem'

Jan  9 16:15:46 mydomain pop3s[11233]: TLS server engine: cannot load cert/key data

Jan  9 16:15:46 mydomain pop3s[11233]: [pop3d] error initializing TLS

Jan  9 16:15:46 mydomain pop3s[11233]: Fatal error: tls_init() failed

Jan  9 16:15:46 mydomain pop3s[11233]: counts: retr=<0> top=<0> dele=<0>

Yes, I bumped into our old posts about Let's Encrypt. I've also made the following change to /etc/postfix/main.cf:

smtpd_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt

smtpd_tls_cert_file = /etc/letsencrypt/live/www.howitts.co.uk/fullchain.pem

smtpd_tls_key_file = /etc/letsencrypt/live/www.howitts.co.uk/privkey.pem

And I have not had to adjust my phone or accept new certificates, so it looks like it is working with STARTTLS. I realised what I was doing wrong in my earlier test.

Thanks Nick.
I have been using Let's Encrypt for only the website as I was scared off doing it for email. I didn't want to have to go to each of our family every couple of months to do the updated certificate install. I've left Let's Encrypt, for the time, implemented using the software from the repo rather than using the Market App.

I figured getting the self signed certificates was the way to go for email.

I will try using the Let's Encrypt certificates and report back.

Nick Howitt wrote:

I'm going to have to check my imap certificates when I get home.

Searching the forum for genkey I notice you are also using Let's Encrypt. In the earlier thread I think I implemented Let's Encrypt in imap incorrectly which is why I was seeing the renewal issue. If you have Let's Encrypt, try putting this in imap.conf:

tls_key_file:    /etc/letsencrypt/live/{your_web_site}/privkey.pem

tls_cert_file:   /etc/letsencrypt/live/{your_web_site}/fullchain.pem

and leave

tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt

I'll give it a go. If it works without giving a prompt to accept the certificate, I set up the automatic renewal to trigger a restart of cyrus-imap.

Yay. That seems to work without being prompted for a certificate in K-9 on my android device.

For the let's Encrypt renewal, I've duplicated the file /var/clearos/events/lets_encrypt/lets_encrypt and changed the contents to:

#!/bin/sh

# created by njh



sleep 10



systemctl condrestart cyrus-imapd.service

(any files I change I add a reference to njh do I can grep for them for future reference.)

I'm going to have to check my imap certificates when I get home.

Searching the forum for genkey I notice you are also using Let's Encrypt. In the earlier thread I think I implemented Let's Encrypt in imap incorrectly which is why I was seeing the renewal issue. If you have Let's Encrypt, try putting this in imap.conf:

tls_key_file:    /etc/letsencrypt/live/{your_web_site}/privkey.pem

tls_cert_file:   /etc/letsencrypt/live/{your_web_site}/fullchain.pem

and leave

tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt

I'll give it a go. If it works without giving a prompt to accept the certificate, I set up the automatic renewal to trigger a restart of cyrus-imap.

Nick,
I created a self-signed certificate using

genkey --days 3650 mail.mydomain.com

It created the .crt and .key files and wrote them to /etc/pki/tls/certs/mail.mydomain.com.crt and /etc/pki/tls/private/mail.mydomain.com.key

Posts crossed. Did genkey create a self-signed certificate or did it use your CA?

It should be possible to use your own files. Just try it. It is easy to revert. Note that if you have your own commercial certificate you may also have a chain/intermediate certificate file. If you do, I need to check up on how to handle it. It may need to be concatenated with your certificate file (I am not sure which order, or if it is relevant. I'll have a look at the Let's Encrypt fullchain file when I get home). Alternatively it can be stuck on the end of the ca-bundle but this is a system file.

The ca-bundle is a list pf all parent CA's which are considered trustworthy. The intermediate certificate links the parent CA to your certificate.

Hmm, this isn't working.
I tried with both .crt and .key files in the format created by genkey. That didn't work.
I changed them both to .pem files using

openssl x509 -in /etc/pki/tls/certs/mail.mydomain.com.crt -out /etc/pki/tls/certs/mail.mydomain.com.crt.pem -outform pem

openssl rsa -in /etc/pki/tls/private/mail.mydomain.key -text > /etc/pki/tls/private/mail.mydomain.key.pem

And that doesn't work either.

Jan 9 10:23:49 mydomain imaps[1237]: TLS server engine: cannot load CA data

Jan 9 10:23:49 mydomain imaps[1237]: unable to get certificate from '/etc/pki/tls/certs/mail.mydomain.com.crt.pem'

Jan 9 10:23:49 mydomain imaps[1237]: TLS server engine: cannot load cert/key data

Jan 9 10:23:49 mydomain imaps[1237]: error initializing TLS

Jan 9 10:23:49 mydomain imaps[1237]: Fatal error: tls_init() failed

Jan 9 10:23:49 mydomain imaps[1243]: TLS server engine: cannot load CA data

Jan 9 10:23:49 mydomain imaps[1243]: unable to get certificate from '/etc/pki/tls/certs/mail.mydomain.com.crt.pem'

Jan 9 10:23:49 mydomain imaps[1243]: TLS server engine: cannot load cert/key data

Jan 9 10:23:49 mydomain imaps[1243]: error initializing TLS

Jan 9 10:23:49 mydomain imaps[1243]: Fatal error: tls_init() failed

I'm going back to the original that was commented in the /etc/imapd.conf file for now.
This is very confusing.

Thank you Nick.
Is it OK to use mail.mydomain.com.crt and mail.mydomain.com.key files rather than .pem files?

The tls_ca_file is /etc/pki/tls/certs/ca-bundle.crt. When I look at it it shows

openssl x509 -in /etc/pki/tls/certs/ca-bundle.crt -inform pem -noout -text

Certificate:

    Data:

        Version: 3 (0x2)

        Serial Number: 6828...800 (0x5ec...4e0)

    Signature Algorithm: sha1WithRSAEncryption

        Issuer: CN=ACCVRAIZ1, OU=PKIACCV, O=ACCV, C=ES

        Validity

            Not Before: May  5 09:37:37 2011 GMT

            Not After : Dec 31 09:37:37 2030 GMT

        Subject: CN=ACCVRAIZ1, OU=PKIACCV, O=ACCV, C=ES

        Subject Public Key Info:

            Public Key Algorithm: rsaEncryption

                Public-Key: (4096 bit)

                Modulus:

                   ....

                Exponent: 65537 (0x10001)

        X509v3 extensions:

            Authority Information Access: 

                CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt

                OCSP - URI:http://ocsp.accv.es



            X509v3 Subject Key Identifier: 

                D2....BD

            X509v3 Basic Constraints: critical

                CA:TRUE

            X509v3 Authority Key Identifier: 

                keyid2...BD



            X509v3 Certificate Policies: 

                Policy: X509v3 Any Policy

                  User Notice:

                    Explicit Text: 

                  CPS: http://www.accv.es/legislacion_c.htm



            X509v3 CRL Distribution Points: 



                Full Name:

                  URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl



            X509v3 Key Usage: critical

                Certificate Sign, CRL Sign

            X509v3 Subject Alternative Name: 

                email:accv@accv.es

    Signature Algorithm: sha1WithRSAEncryption

         ....

This is not what I expected. Isn't this supposed to be the cert for my box? Or Is it a cyrus std certificate?

Thanks again.

Hi Nuke,
You will need to point your /etc/imap.conf files to your new certificates. I think genkey creates a self-signed one so you just get a certificate and key. Comment out the three tls entries in imap.conf (so it is easier to revert) and add your own lines pointing to your certificate and key (don't overwrite the originals). With a self-signed certificate you may not need to comment out the tls_ca_file setting.

Note you should be able to use your system certificates if you want instead by adding the lines:

tls_cert_file: /etc/pki/CA/sys-0-cert.pem

tls_key_file: /etc/pki/CA/private/sys-0-key.pem

tls_ca_file: /etc/pki/CA/ca-cert.pem

Restart cyrus-imapd after making changes.