1 Topic by Wraptor (edited by Wraptor 2017-03-14 23:49:32)

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Topic: Sending mails from smtp username and aliases specified only

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.6
- Linux/BSD distribution name and version: Debian Jessie 8.7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Apache2
- Manage mail accounts with iRedAdmin-Pro? Nope
- Related log if you're reporting an issue:
====

Dear forum users

I've setup my iRedMail server and I've setup all domains etc.
For testing purposes I've tried sending mails with the specified aliases, which worked.
I also tried sending mails with non existing mail aliases, which worked aswell.

I'm trying to achieve the following: Allow the user to send mails from his smtp auth username aswell as his aliases, but only his specified aliases.

This is what I've got using the following tutorial:
The 'reject_sender_login_mismatch' has been removed from the main.cf file (Actually it was never there).
The /opt/iredapd/settings.py:

ALLOWED_LOGIN_MISMATCH_STRICTLY = True & plugins = ["reject_null_sender", "reject_sender_login_mismatch", "greylisting", "throttle", "amavisd_wblist", "sql_alias_access_policy"]

I've restarted all servers aswell as rebooting the VM.
Can anybody help me out?

Thanks in advance,
Wraptor

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

I'm trying to achieve the following: Allow the user to send mails from his smtp auth username aswell as his aliases, but only his specified aliases.

This is enabled with default iRedAPD setting. What's the error message did you get while trying to send as the per-user alias address?

Please turn on debug mode in iRedAPD and reproduce this issue, extract full log related to your testing and paste here.
FYI: http://www.iredmail.org/docs/debug.iredapd.html

3 Reply by Wraptor (edited by Wraptor 2017-03-15 02:02:56)

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

Dear ZhangHuangbin

Everything works, I can send mails. But as any random alias. Not the specified ones as ALLOWED_LOGIN_MISMATCH_STRICTLY = True should enforce

Edit: I've redone my tests and here's a sample of the iredapd.log file in debug mode:

SELECT alias_domain
                           FROM alias_domain
                          WHERE alias_domain='domain.com' AND target_domain='domain.com'
                          LIMIT 1
2017-03-14 18:43:37 DEBUG SQL query result: None
2017-03-14 18:43:37 DEBUG No alias domain found.
2017-03-14 18:43:37 DEBUG <-- Result: REJECT Sender is not same as SMTP authenticate username
2017-03-14 18:43:37 DEBUG Session ended.
2017-03-14 18:43:37 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx RCPT, user@domain.com => usersalias@domain.com -> gmailuser@gmail.com, REJECT Sender is not same as SMTP authenticate username [0.0009s]

Regards
Wraptor

4 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

I'm trying to use aliases from the alias table, aswell as the alias_domain table. Even though iredapd only checks alias_domain as far as the debug log shows

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

Not the specified ones as ALLOWED_LOGIN_MISMATCH_STRICTLY = True should enforce

If you check file /opt/iredapd/libs/default_settings.py, this is the default setting.

Wraptor wrote:

I'm trying to use aliases from the alias table, aswell as the alias_domain table. Even though iredapd only checks alias_domain as far as the debug log shows

Please show me the SQL records related to this user and alias. I need to figure out the relationship between user and the alias.

6 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

Here's are SQL records from the alias table: http://imgur.com/a/6rtCB
alias_domain table: http://imgur.com/b10j8Sa

I've noticed that I'm not able to send emails as not existing aliases using a custom build debug SMTP tool in C# but outlook allows me to send as any alias. Strange...

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

I'm afraid that i'm lost. Please help me understand this issue with steps below:

- Show me the email address you used to for smtp auth. You can replace the real domain name by, for example, mydomain.com.
- Show me the email address you want to sent as different user. (random alias?)
- Show me the SQL records of above 2 email addresses, so that i can know the relationship between them.
- Turn on debug mode in iRedAPD. Send testing email to trigger the issue, show me FULL log related to your testing email. Again, you can replace the real domain names, but different real domain names should be replaced by different fake domain names, otherwise i will be lost again.

8 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

I've redone some testing to show you the results and here's what I've got using a custom tool.
Source code (C#):

static void Main(string[] args)
{
    Console.Write("Server: ");
    string Server = Console.ReadLine();
    Console.Write("Auth username: ");
    string Username = Console.ReadLine();
    Console.Write("Auth password: ");
    string Password = Console.ReadLine();
    while(true)
    {
        Console.WriteLine("Re-enter credentials? (y/n)");
        if(Console.ReadLine().ToLower().StartsWith("y"))
            Main(args);


        Console.Write("Sender address: ");
        string Address = Console.ReadLine();
        Console.Write("To address: ");
        string To = Console.ReadLine();
        try
        {
            MailMessage mail = new MailMessage(Address, To);
            mail.Subject = "";
            mail.Body = "";

            SmtpClient Client = new SmtpClient();
            Client.UseDefaultCredentials = false;
            Client.Credentials = new NetworkCredential(Username, Password);
            Client.DeliveryMethod = SmtpDeliveryMethod.Network;
            Client.Host = Server;
            Client.Port = 587;
            Client.EnableSsl = true;

            Client.Send(mail);

            Client.Dispose();
            Client = null;
        }
        catch (SmtpException ex)
        {
            Console.WriteLine(ex.Message);
        }
    }
}

I'll be authenticating using the email address: wraptor@example.com
There is an alias hostmaster@example.com pointed to the user wraptor@example.com
MySQL alias row with this alias:

+---------------------------------+-----------------------------+------------------+-----------+-----------------------------+---------+
| address                            | goto                             | domain           | is_alias | alias_to                        | active |
+---------------------------------+-----------------------------+------------------+-----------+-----------------------------+---------+
| hostmaster@example.com | wraptor@example.com | example.com |           1 | wraptor@example.com |        1 |
+---------------------------------+-----------------------------+------------------+-----------+-----------------------------+---------+

Result using my SMTP tool:

Transaction failed. The server response was: 5.7.1 <blabla@gmail.com>: Recipient address rejected: Sender is not same as SMTP authenticate username

Using outlook I can send with any alias: http://imgur.com/a/E3zX3
I litterally used notexising as alias and used Google developer console to filter for privacy.

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

I got your point: iRedAPD plugin should check per-user alias address, right?

10 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

ZhangHuangbin wrote:

I got your point: iRedAPD plugin should check per-user alias address, right?

Yes, but the problem is: how? Or isn't this how the feature works?

I'm new to mailservers on Linux as I'm used to run a mailserver with hMailserver on Windows Server 2016.
I have to say though: iRedMail is way faster, more features but harder to configure as you require ssh and do not have any fancy UI of some sort (Appart from the very limited non pro admin panel).

But isn't this supposed to be a feature that is requested a lot?

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

Yes, but the problem is: how? Or isn't this how the feature works?

Our plugin doesn't implement this. But don't worry, here's patch to fix it, please help test it:
https://bitbucket.org/zhb/iredapd/commi … bce1ad0975

Wraptor wrote:

I have to say though: iRedMail is way faster, more features but harder to configure as you require ssh and do not have any fancy UI of some sort (Appart from the very limited non pro admin panel).

That's how most linux softwares working -- especially server softwares.
About the iRedAdmin-Pro, it will make your life much easier, and your purchase will keep iRedMail project alive. smile

Wraptor wrote:

But isn't this supposed to be a feature that is requested a lot?

Don't "suppose", feel free to ask / request.

12 Reply by Wraptor (edited by Wraptor 2017-03-17 02:48:40)

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

This indeed successfully allowed me to send messages using per-user aliases.
The problem is that I can now send as any alias as long as it exists.
Even with other mailbox addresses (Which aren't aliases of the user wraptor@example.com, the one I authenticated with on smtp).

2017-03-15 16:46:03 DEBUG No sender throttle setting.
2017-03-15 16:46:03 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-15 16:46:03 DEBUG <-- Result: DUNNO
2017-03-15 16:46:03 DEBUG Skip plugin: sql_alias_access_policy (protocol_state != END-OF-MESSAGE)
2017-03-15 16:46:03 DEBUG Skip plugin: amavisd_wblist (protocol_state != END-OF-MESSAGE)
2017-03-15 16:46:03 DEBUG Session ended.
2017-03-15 16:46:03 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx END-OF-MESSAGE, wraptor@example.com => noreply@example.com -> blabla@gmail.com, DUNNO [0.0009s]

noreply@example.com is a mailbox:

+-----------------------------+------------+------------------+
| username                     | name      | domain          |
+-----------------------------+------------+------------ -----+
| noreply@example.com | No-reply | example.com |
+-----------------------------+------------+------------------+

wraptor@example.com is not in the ALLOWED_LOGIN_MISMATCH_SENDERS list (The list isn't specified at all).

Current iredapd config file:

...Default config (mysql login etc)...

# Login mismatch

ALLOWED_LOGIN_MISMATCH_STRICTLY = True

# Greylistening
GREYLISTING_MESSAGE = "Please try again later."
GREYLISTING_AUTH_TRIPLET_EXPIRE = 30
GREYLISTING_UNAUTH_TRIPLET_EXPIRE = 1
GREYLISTING_BLOCK_EXPIRE = 5


Edit: Also I would appreciate you removing my email address from the bitbucket credits :Þ
"Thanks Wraptor <personalemail> for the report in forum."

I appreciate the credits for requesting the 'feature' but I'd like to keep this mail safe for spammers etc. :Þ

13 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

SQL command in above patch is not correct, please also apply this patch (based on first patch):
https://bitbucket.org/zhb/iredapd/commi … 441c0c814e

Wraptor wrote:

The problem is that I can now send as any alias as long as it exists.
Even with other mailbox addresses (Which aren't aliases of the user wraptor@example.com, the one I authenticated with on smtp).

AGAIN, show me FULL iredapd log related to your testing email. The log you pasted is not related to this iredapd plugin AT ALL.

14 Reply by Wraptor (edited by Wraptor 2017-03-16 00:15:20)

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

comfore commit update

2017-03-15 16:46:03 DEBUG [SQL] Query result:
[]
2017-03-15 16:46:03 DEBUG No sender throttle setting.
2017-03-15 16:46:03 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-15 16:46:03 DEBUG <-- Result: DUNNO
2017-03-15 16:46:03 DEBUG Skip plugin: sql_alias_access_policy (protocol_state != END-OF-MESSAGE)
2017-03-15 16:46:03 DEBUG Skip plugin: amavisd_wblist (protocol_state != END-OF-MESSAGE)
2017-03-15 16:46:03 DEBUG Session ended.
2017-03-15 16:46:03 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx END-OF-MESSAGE, wraptor@example.com => noreply@example.com -> blabla@gmail.com, DUNNO [0.0009s]

After incorrect sql query (with iredapd restarted):

2017-03-15 17:06:08 ERROR <!> Error while applying plugin "reject_sender_login_mismatch": ['Traceback (most recent call last):\n', '  File "/opt/iRedAPD-2.0/libs/utils.py", line 101, in apply_plugin\n', '  File "/opt/iredapd/plugins/reject_sender_login_mismatch.py", line 253, in restriction\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 721, in execute\n    return self._execute_text(object, multiparams, params)\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 870, in _execute_text\n    statement, parameters\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context\n    context)\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1159, in _handle_dbapi_exception\n    exc_info\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb)\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context\n    context)\n', '  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute\n    cursor.execute(statement, parameters)\n', '  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute\n    self.errorhandler(self, exc, value)\n', '  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler\n    raise errorclass, errorvalue\n', 'OperationalError: (OperationalError) (1054, "Unknown column \'isalias\' in \'where clause\'") "SELECT address\\n                           FROM alias\\n                          WHERE address=\'noreply@example.com\' AND alias_to=\'wraptor@example.com\' AND isalias=1\\n                          LIMIT 1" ()\n']
2017-03-15 17:06:08 DEBUG --> Apply plugin: greylisting
2017-03-15 17:06:08 DEBUG Found SASL username, bypass greylisting for outbound email.
2017-03-15 17:06:08 DEBUG <-- Result: DUNNO
2017-03-15 17:06:08 DEBUG --> Apply plugin: throttle
2017-03-15 17:06:08 DEBUG Check sender throttling.
2017-03-15 17:06:08 DEBUG [SQL] Query throttle setting:

        SELECT id, account, priority, period, max_msgs, max_quota, msg_size
          FROM throttle
         WHERE kind='outbound' AND account IN ('2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx', '@ip', '@.', 'wraptor@example.com', '@example.com', '@.example.com, '@com', '@.com')
         ORDER BY priority DESC

2017-03-15 17:06:08 DEBUG [SQL] Query result:
[]
2017-03-15 17:06:08 DEBUG No sender throttle setting.
2017-03-15 17:06:08 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-15 17:06:08 DEBUG <-- Result: DUNNO
2017-03-15 17:06:08 DEBUG --> Apply plugin: sql_alias_access_policy
2017-03-15 17:06:08 DEBUG [SQL] query access policy:
SELECT accesspolicy, goto, moderators
               FROM alias
              WHERE address='blabla@gmail.com'
                    AND islist=1
                    AND active=1
              LIMIT 1

2017-03-15 17:06:08 DEBUG SQL query result: None
2017-03-15 17:06:08 DEBUG [SQL] Check whether recipient domain is an alias domain:
SELECT target_domain
                   FROM alias_domain
                  WHERE alias_domain = 'gmail.com'
                  LIMIT 1

2017-03-15 17:06:08 DEBUG [SQL] query result: None
2017-03-15 17:06:08 DEBUG Recipient domain is not an alias domain.
2017-03-15 17:06:08 DEBUG <-- Result: DUNNO Recipient is not a mail alias account or no access policy
2017-03-15 17:06:08 DEBUG --> Apply plugin: amavisd_wblist
2017-03-15 17:06:08 DEBUG Possible policy senders: ['@.', 'wraptor@example.com', '@example.com', '@.example.com', '@com', '@.com, '2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx']
2017-03-15 17:06:08 DEBUG Possible policy recipients: ['@.', 'blabla@gmail.com', '@gmail.com', '@.gmail.com', '@com', '@.com']
2017-03-15 17:06:08 DEBUG Apply wblist for outbound message.
2017-03-15 17:06:08 DEBUG [SQL] Query local addresses:
SELECT id, email
               FROM users
              WHERE email IN ('@.', 'wraptor@example.com', '@example.com', '@.example.com', '@com', '@.com', '2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx')
           ORDER BY priority DESC
2017-03-15 17:06:08 DEBUG No record found in SQL database.
2017-03-15 17:06:08 DEBUG No valid sender id or recipient id.
2017-03-15 17:06:08 DEBUG [SQL] query local domain (gmail.com):
SELECT domain
                   FROM domain
                  WHERE domain='gmail.com'
                  LIMIT 1
2017-03-15 17:06:08 DEBUG SQL query result: None
2017-03-15 17:06:08 DEBUG [SQL] query alias domains (gmail.com):
SELECT alias_domain
                   FROM alias_domain
                  WHERE alias_domain='gmail.com' OR target_domain='gmail.com'
                  LIMIT 1
2017-03-15 17:06:08 DEBUG SQL query result: None
2017-03-15 17:06:08 DEBUG <-- Result: DUNNO
2017-03-15 17:06:08 DEBUG Session ended.
2017-03-15 17:06:08 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx RCPT, wraptor@example.com => noreply@example.com -> blabla@gmail.com, DUNNO [0.0266s]
2017-03-15 17:06:08 DEBUG smtp session: request=smtpd_access_policy
2017-03-15 17:06:08 DEBUG smtp session: protocol_state=END-OF-MESSAGE
2017-03-15 17:06:08 DEBUG smtp session: protocol_name=ESMTP
2017-03-15 17:06:08 DEBUG smtp session: client_address=2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx
2017-03-15 17:06:08 DEBUG smtp session: client_name=unknown
2017-03-15 17:06:08 DEBUG smtp session: reverse_client_name=XXXXX
2017-03-15 17:06:08 DEBUG smtp session: helo_name=WC2016S10
2017-03-15 17:06:08 DEBUG smtp session: sender=noreply@example.com
2017-03-15 17:06:08 DEBUG smtp session: recipient=blabla@gmail.com
2017-03-15 17:06:08 DEBUG smtp session: recipient_count=1
2017-03-15 17:06:08 DEBUG smtp session: queue_id=4D09C20F9E
2017-03-15 17:06:08 DEBUG smtp session: instance=2c3.58c9666f.ef2de.0
2017-03-15 17:06:08 DEBUG smtp session: size=113
2017-03-15 17:06:08 DEBUG smtp session: etrn_domain=
2017-03-15 17:06:08 DEBUG smtp session: stress=
2017-03-15 17:06:08 DEBUG smtp session: sasl_method=login
2017-03-15 17:06:08 DEBUG smtp session: sasl_username=wraptor@example.com
2017-03-15 17:06:08 DEBUG smtp session: sasl_sender=
2017-03-15 17:06:08 DEBUG smtp session: ccert_subject=
2017-03-15 17:06:08 DEBUG smtp session: ccert_issuer=
2017-03-15 17:06:08 DEBUG smtp session: ccert_fingerprint=
2017-03-15 17:06:08 DEBUG smtp session: ccert_pubkey_fingerprint=
2017-03-15 17:06:08 DEBUG smtp session: encryption_protocol=TLSv1
2017-03-15 17:06:08 DEBUG smtp session: encryption_cipher=ECDHE-RSA-AES256-SHA
2017-03-15 17:06:08 DEBUG smtp session: encryption_keysize=256
2017-03-15 17:06:08 DEBUG Skip plugin: reject_null_sender (protocol_state != END-OF-MESSAGE)
2017-03-15 17:06:08 DEBUG Skip plugin: reject_sender_login_mismatch (protocol_state != END-OF-MESSAGE)
2017-03-15 17:06:08 DEBUG Skip plugin: greylisting (protocol_state != END-OF-MESSAGE)
2017-03-15 17:06:08 DEBUG --> Apply plugin: throttle
2017-03-15 17:06:08 DEBUG Check sender throttling.
2017-03-15 17:06:08 DEBUG [SQL] Query throttle setting:

        SELECT id, account, priority, period, max_msgs, max_quota, msg_size
          FROM throttle
         WHERE kind='outbound' AND account IN ('2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx', '@ip', '@.', 'wraptor@example.com', '@example.com', '@.example.com', '@com', '@.com')
         ORDER BY priority DESC

2017-03-15 17:06:08 DEBUG [SQL] Query result:
[]
2017-03-15 17:06:08 DEBUG No sender throttle setting.
2017-03-15 17:06:08 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-15 17:06:08 DEBUG <-- Result: DUNNO
2017-03-15 17:06:08 DEBUG Skip plugin: sql_alias_access_policy (protocol_state != END-OF-MESSAGE)
2017-03-15 17:06:08 DEBUG Skip plugin: amavisd_wblist (protocol_state != END-OF-MESSAGE)
2017-03-15 17:06:08 DEBUG Session ended.
2017-03-15 17:06:08 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxx END-OF-MESSAGE, wraptor@example.com => noreply@example.com -> blabla@gmail.com, DUNNO [0.0143s]

Also would you mind reading and solving my edit on my comment above?

15 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

I've found the error with the SQL request, the query must be is_alias not isalias.

16 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Fixed. You can fix it manually and restart iredapd, then test again. Please show us related debug log if it doesn't work.

17 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

Everything works as far as I can see. One thing I'd like to note is that if you use an alias that does not belong to the sender the server will reply with:

The server response was: 5.7.1 <blablka@gmail.com>: Recipient address rejected: Sender is not same as SMTP authenticate username

And also the debug log only outputs: "[]" if you successfully found an alias:

2017-03-16 17:24:17 DEBUG [SQL] Query result: []


Also whenever I use Microsoft Outlook 2016 I can send emails as any non exisiting aliases.

How SMTP should react to this user:

Sender address: nonexistinguser@example.com
To address: blablabla@gmail.com
Transaction failed. The server response was: 5.7.1 <blablabla@gmail.com>: Recipient address rejected: Sender is not same as SMTP authenticate username

Information I entered (Sender is alias using wraptor@example.com SMTPauthentication): http://imgur.com/a/TM1t7
What happened: http://imgur.com/a/vRo0t

iRedAPD logs

2017-03-16 17:24:17 DEBUG No sender throttle setting.
2017-03-16 17:24:17 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG Skip plugin: sql_alias_access_policy (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Skip plugin: amavisd_wblist (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Session ended.
2017-03-16 17:24:17 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx END-OF-MESSAGE, wraptor@example.com => blablabla@gmail.com, DUNNO [0.0009s]

/var/log/mail.log file

Mar 16 17:24:17 uranus postfix/qmgr[2102]: CB50D211CA: from=<wraptor@example.com>, size=3796, nrcpt=1 (queue active)
Mar 16 17:24:17 uranus postfix/10025/smtpd[4114]: disconnect from localhost[127.0.0.1]
Mar 16 17:24:17 uranus amavis[2104]: (02104-02) Passed CLEAN {RelayedInternal}, ORIGINATING LOCAL [2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:61298 [2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx] <wraptor@example.com> -> <blablabla@gmail.com>, Queue-ID: 6D378211C9, Message-ID: <000e01d29e71$c23a2810$46ae7830$@example.com>, mail_id: dtgCuufCLEHt, Hits: -0.997, size: 2715, queued_as: CB50D211CA, dkim_new=dkim:example.com, 381 ms, Tests: [ALL_TRUSTED=-1,HEADER_FROM_DIFFERENT_DOMAINS=0.001,HTML_MESSAGE=0.001,MIME_HTML_MOSTLY=0.001]
Mar 16 17:24:17 uranus postfix/smtp-amavis/smtp[4111]: 6D378211C9: to=<blablabla@gmail.com>, relay=127.0.0.1[127.0.0.1]:10026, delay=0.54, delays=0.09/0/0.01/0.44, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as CB50D211CA)
Mar 16 17:24:17 uranus postfix/qmgr[2102]: 6D378211C9: removed
Mar 16 17:24:20 uranus postfix/submission/smtpd[4098]: disconnect from unknown[2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]
Mar 16 17:24:47 uranus postfix/smtp[4115]: connect to gmail-smtp-in.l.google.com[2a00:1450:4013:c04::1a]:25: Connection timed out
Mar 16 17:24:48 uranus postfix/smtp[4115]: Trusted TLS connection established to gmail-smtp-in.l.google.com[xxx.xxx.143.27]:25: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
Mar 16 17:24:48 uranus postfix/smtp[4115]: CB50D211CA: to=<blablabla@gmail.com>, relay=gmail-smtp-in.l.google.com[xxx.xxx.143.27]:25, delay=31, delays=0.06/0.02/30/0.25, dsn=2.0.0, status=sent (250 2.0.0 OK 1489681488 t144si5166228wmt.46 - gsmtp)
Mar 16 17:24:48 uranus postfix/qmgr[2102]: CB50D211CA: removed


What can I do, as iredapd doesn't see the alias it's being send as in the case of outlook.

18 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

And also the debug log only outputs: "[]" if you successfully found an alias:

You really should paste FULL FULL FULL FULL log related to the testing email. i cannot understand where this log line was logged by iRedAPD.

Wraptor wrote:

Also whenever I use Microsoft Outlook 2016 I can send emails as any non exisiting aliases.

iRedAPD gets only data sent by Postfix, if you turn on debug mode (again, you didn't paste FULL FULL FULL debug log), you will see iredapd log line which contains 'smtp session: xx=yy', it will show you all data it got from Postfix.

A known "issue" is that Postfix didn't send the sender address in mail message to iRedAPD, this is expected, because Postfix just get data of SMTP session sent by your server and sender server.

19 Reply by Wraptor (edited by Wraptor 2017-03-18 01:18:31)

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

FULL FULL FULL FULL /var/log/mail.log part

Mar 16 17:24:17 uranus postfix/submission/smtpd[4098]: connect from unknown[2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]
Mar 16 17:24:17 uranus postfix/submission/smtpd[4098]: Anonymous TLS connection established from unknown[2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Mar 16 17:24:17 uranus postfix/submission/smtpd[4098]: 6D378211C9: client=unknown[2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx], sasl_method=LOGIN, sasl_username=wraptor@example.com
Mar 16 17:24:17 uranus postfix/cleanup[4106]: 6D378211C9: message-id=<000e01d29e71$c23a2810$46ae7830$@example.com>
Mar 16 17:24:17 uranus postfix/qmgr[2102]: 6D378211C9: from=<wraptor@example.com>, size=2715, nrcpt=1 (queue active)
Mar 16 17:24:17 uranus postfix/10025/smtpd[4114]: connect from localhost[127.0.0.1]
Mar 16 17:24:17 uranus postfix/10025/smtpd[4114]: CB50D211CA: client=localhost[127.0.0.1]
Mar 16 17:24:17 uranus postfix/cleanup[4106]: CB50D211CA: message-id=<000e01d29e71$c23a2810$46ae7830$@example.com>
Mar 16 17:24:17 uranus postfix/qmgr[2102]: CB50D211CA: from=<wraptor@example.com>, size=3796, nrcpt=1 (queue active)
Mar 16 17:24:17 uranus postfix/10025/smtpd[4114]: disconnect from localhost[127.0.0.1]
Mar 16 17:24:17 uranus amavis[2104]: (02104-02) Passed CLEAN {RelayedInternal}, ORIGINATING LOCAL [2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:61298 [2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx] <wraptor@example.com> -> <blablabla@gmail.com>, Queue-ID: 6D378211C9, Message-ID: <000e01d29e71$c23a2810$46ae7830$@example.com>, mail_id: dtgCuufCLEHt, Hits: -0.997, size: 2715, queued_as: CB50D211CA, dkim_new=dkim:example.com, 381 ms, Tests: [ALL_TRUSTED=-1,HEADER_FROM_DIFFERENT_DOMAINS=0.001,HTML_MESSAGE=0.001,MIME_HTML_MOSTLY=0.001]
Mar 16 17:24:17 uranus postfix/smtp-amavis/smtp[4111]: 6D378211C9: to=<blablabla@gmail.com>, relay=127.0.0.1[127.0.0.1]:10026, delay=0.54, delays=0.09/0/0.01/0.44, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as CB50D211CA)
Mar 16 17:24:17 uranus postfix/qmgr[2102]: 6D378211C9: removed
Mar 16 17:24:20 uranus postfix/submission/smtpd[4098]: disconnect from unknown[2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]
Mar 16 17:24:47 uranus postfix/smtp[4115]: connect to gmail-smtp-in.l.google.com[2a00:1450:4013:c04::1a]:25: Connection timed out
Mar 16 17:24:48 uranus postfix/smtp[4115]: Trusted TLS connection established to gmail-smtp-in.l.google.com[74.125.143.27]:25: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
Mar 16 17:24:48 uranus postfix/smtp[4115]: CB50D211CA: to=<blablabla@gmail.com>, relay=gmail-smtp-in.l.google.com[74.125.143.27]:25, delay=31, delays=0.06/0.02/30/0.25, dsn=2.0.0, status=sent (250 2.0.0 OK 1489681488 t144si5166228wmt.46 - gsmtp)
Mar 16 17:24:48 uranus postfix/qmgr[2102]: CB50D211CA: removed

FULL FULL FULL FULL /var/log/iredapd/iredapd.log

2017-03-16 17:24:17 DEBUG Connect from 127.0.0.1, port 55199.
2017-03-16 17:24:17 DEBUG smtp session: request=smtpd_access_policy
2017-03-16 17:24:17 DEBUG smtp session: protocol_state=RCPT
2017-03-16 17:24:17 DEBUG smtp session: protocol_name=ESMTP
2017-03-16 17:24:17 DEBUG smtp session: client_address=2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
2017-03-16 17:24:17 DEBUG smtp session: client_name=unknown
2017-03-16 17:24:17 DEBUG smtp session: reverse_client_name=unknown
2017-03-16 17:24:17 DEBUG smtp session: helo_name=CatchMeOutsideHowBowDah
2017-03-16 17:24:17 DEBUG smtp session: sender=wraptor@example.com
2017-03-16 17:24:17 DEBUG smtp session: recipient=blablabla@gmail.com
2017-03-16 17:24:17 DEBUG smtp session: recipient_count=0
2017-03-16 17:24:17 DEBUG smtp session: queue_id=
2017-03-16 17:24:17 DEBUG smtp session: instance=1002.58cabc31.67c51.0
2017-03-16 17:24:17 DEBUG smtp session: size=0
2017-03-16 17:24:17 DEBUG smtp session: etrn_domain=
2017-03-16 17:24:17 DEBUG smtp session: stress=
2017-03-16 17:24:17 DEBUG smtp session: sasl_method=LOGIN
2017-03-16 17:24:17 DEBUG smtp session: sasl_username=wraptor@example.com
2017-03-16 17:24:17 DEBUG smtp session: sasl_sender=
2017-03-16 17:24:17 DEBUG smtp session: ccert_subject=
2017-03-16 17:24:17 DEBUG smtp session: ccert_issuer=
2017-03-16 17:24:17 DEBUG smtp session: ccert_fingerprint=
2017-03-16 17:24:17 DEBUG smtp session: ccert_pubkey_fingerprint=
2017-03-16 17:24:17 DEBUG smtp session: encryption_protocol=TLSv1.2
2017-03-16 17:24:17 DEBUG smtp session: encryption_cipher=ECDHE-RSA-AES256-GCM-SHA384
2017-03-16 17:24:17 DEBUG smtp session: encryption_keysize=256
2017-03-16 17:24:17 DEBUG --> Apply plugin: reject_null_sender
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG --> Apply plugin: reject_sender_login_mismatch
2017-03-16 17:24:17 DEBUG Sender: wraptor@example.com, SASL username: wraptor@example.com
2017-03-16 17:24:17 DEBUG SKIP: sender == sasl username.
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG --> Apply plugin: greylisting
2017-03-16 17:24:17 DEBUG Found SASL username, bypass greylisting for outbound email.
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG --> Apply plugin: throttle
2017-03-16 17:24:17 DEBUG Check sender throttling.
2017-03-16 17:24:17 DEBUG [SQL] Query throttle setting:
2017-03-16 17:24:17 DEBUG [SQL] Query result:
2017-03-16 17:24:17 DEBUG No sender throttle setting.
2017-03-16 17:24:17 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG --> Apply plugin: sql_alias_access_policy
2017-03-16 17:24:17 DEBUG [SQL] query access policy:
2017-03-16 17:24:17 DEBUG SQL query result: None
2017-03-16 17:24:17 DEBUG [SQL] Check whether recipient domain is an alias domain:
2017-03-16 17:24:17 DEBUG [SQL] query result: None
2017-03-16 17:24:17 DEBUG Recipient domain is not an alias domain.
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO Recipient is not a mail alias account or no access policy
2017-03-16 17:24:17 DEBUG --> Apply plugin: amavisd_wblist
2017-03-16 17:24:17 DEBUG Possible policy senders: ['@.', 'wraptor@example.com', '@example.com', '@.example.com', '@com', '@.com', '2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx']
2017-03-16 17:24:17 DEBUG Possible policy recipients: ['@.', 'blablabla@gmail.com', '@gmail.com', '@.gmail.com', '@com', '@.com']
2017-03-16 17:24:17 DEBUG Apply wblist for outbound message.
2017-03-16 17:24:17 DEBUG [SQL] Query local addresses:
2017-03-16 17:24:17 DEBUG No record found in SQL database.
2017-03-16 17:24:17 DEBUG No valid sender id or recipient id.
2017-03-16 17:24:17 DEBUG [SQL] query local domain (gmail.com):
2017-03-16 17:24:17 DEBUG SQL query result: None
2017-03-16 17:24:17 DEBUG [SQL] query alias domains (gmail.com):
2017-03-16 17:24:17 DEBUG SQL query result: None
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG Session ended.
2017-03-16 17:24:17 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx RCPT, wraptor@example.com => blablabla@gmail.com, DUNNO [0.0104s]
2017-03-16 17:24:17 DEBUG smtp session: request=smtpd_access_policy
2017-03-16 17:24:17 DEBUG smtp session: protocol_state=END-OF-MESSAGE
2017-03-16 17:24:17 DEBUG smtp session: protocol_name=ESMTP
2017-03-16 17:24:17 DEBUG smtp session: client_address=2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
2017-03-16 17:24:17 DEBUG smtp session: client_name=unknown
2017-03-16 17:24:17 DEBUG smtp session: reverse_client_name=unknown
2017-03-16 17:24:17 DEBUG smtp session: helo_name=CatchMeOutsideHowBowDah
2017-03-16 17:24:17 DEBUG smtp session: sender=wraptor@example.com
2017-03-16 17:24:17 DEBUG smtp session: recipient=blablabla@gmail.com
2017-03-16 17:24:17 DEBUG smtp session: recipient_count=1
2017-03-16 17:24:17 DEBUG smtp session: queue_id=6D378211C9
2017-03-16 17:24:17 DEBUG smtp session: instance=1002.58cabc31.67c51.0
2017-03-16 17:24:17 DEBUG smtp session: size=2505
2017-03-16 17:24:17 DEBUG smtp session: etrn_domain=
2017-03-16 17:24:17 DEBUG smtp session: stress=
2017-03-16 17:24:17 DEBUG smtp session: sasl_method=LOGIN
2017-03-16 17:24:17 DEBUG smtp session: sasl_username=wraptor@example.com
2017-03-16 17:24:17 DEBUG smtp session: sasl_sender=
2017-03-16 17:24:17 DEBUG smtp session: ccert_subject=
2017-03-16 17:24:17 DEBUG smtp session: ccert_issuer=
2017-03-16 17:24:17 DEBUG smtp session: ccert_fingerprint=
2017-03-16 17:24:17 DEBUG smtp session: ccert_pubkey_fingerprint=
2017-03-16 17:24:17 DEBUG smtp session: encryption_protocol=TLSv1.2
2017-03-16 17:24:17 DEBUG smtp session: encryption_cipher=ECDHE-RSA-AES256-GCM-SHA384
2017-03-16 17:24:17 DEBUG smtp session: encryption_keysize=256
2017-03-16 17:24:17 DEBUG Skip plugin: reject_null_sender (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Skip plugin: reject_sender_login_mismatch (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Skip plugin: greylisting (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG --> Apply plugin: throttle
2017-03-16 17:24:17 DEBUG Check sender throttling.
2017-03-16 17:24:17 DEBUG [SQL] Query throttle setting:
2017-03-16 17:24:17 DEBUG [SQL] Query result:
2017-03-16 17:24:17 DEBUG No sender throttle setting.
2017-03-16 17:24:17 DEBUG Bypass recipient throttling (found sasl_username).
2017-03-16 17:24:17 DEBUG <-- Result: DUNNO
2017-03-16 17:24:17 DEBUG Skip plugin: sql_alias_access_policy (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Skip plugin: amavisd_wblist (protocol_state != END-OF-MESSAGE)
2017-03-16 17:24:17 DEBUG Session ended.
2017-03-16 17:24:17 INFO 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx END-OF-MESSAGE, wraptor@example.com => blablabla@gmail.com, DUNNO [0.0030s]

Also would it be possible to make a system that creates a new log file every day so debugging/reading through logs wouldn't be as difficult as it is right now?

20 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

2017-03-16 17:24:17 DEBUG smtp session: sender=wraptor@example.com
...
2017-03-16 17:24:17 DEBUG smtp session: sasl_username=wraptor@example.com

In iRedAPD log file, you can see iRedAPD got same sender and sasl username. So the email gone through.

Wraptor wrote:

Also would it be possible to make a system that creates a new log file every day so debugging/reading through logs wouldn't be as difficult as it is right now?

How about use 'grep '<month> <day>' /var/log/maillog' to get the log?

21 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

Using grep on big files might take a while of the log file size gets bigger than lets say a gigabyte, using different log files for each day also allows me to check file sizes as a comparison of traffic etc.

My question about iRedAPD getting trough is how to fix it. I already assumed something like this was happening with outlook setting the sender as that address, but received emails (e.g. in google) are being shown as sent from the alias, that alias can be whatever the user wants it to be defeating the purpose of my per mailbox specific aliases.

Here's a quick decodation/beatification of a packet sent to my gmail account using my mail and an alias using Wireshark.

From: <nonexistinguser@example.com>
   Sender: "Wraptor" <wraptor@example.com>
   To: <blablabla@gmail.com>
   Subject: Test
   Date: Sat, 18 Mar  2017 09:29:55 +0100
   Message-ID: <004801d29fc1$d29e2610$77da7230$@example.com>
   MIME-Version: 1.0
   Content-Type: multipart/alternative;
   boundary="----=_NextPart_000_0049_01D29FCA.34630340"
   X-Mailer: Microsoft Outlook 16.0
   Thread-Index: AdKfwcqseTXXXXXXXXqnrGO6nxs+dQ==
   Content-Language: en-us
   
   This is a multipart message in MIME format.
   ------=_NextPart_000_0049_01D29FCA.34630340
   Content-Type: text/plain;
   charset="us-ascii"
   Content-Transfer-Encoding: 7bit

I've found out that there's another header within the message (DATA) that outlook sends, most to all mailservers will read that one out aswell. Maybe you can modify iRedAPD to check this aswell? I'll try my best to code my own solution but I'm not familliar with Python at all.

22 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Just want to repeat myself: iRedAPD just gets the data sent from Postfix, and the data is just collected from SMTP session (for example, sender server sends a "HELO" smtp command, and Postfix gets it), Postfix doesn't send any data of mail message to iRedAPD.

How iRedAPD works depends on how Postfix calls iRedAPD, it's clearly documented in Postfix website:
http://www.postfix.org/SMTPD_POLICY_README.html

To get mail message info, we need to modify iRedAPD to work as a Postfix milter program, not a policy server. this is a lot work to do, and nearly completely rewritten.

23 Reply by Wraptor

  • Wraptor
  • Member
  • Offline
  • Registered: 2017-03-14
  • Posts: 65

Re: Sending mails from smtp username and aliases specified only

So thats a no go? Guess I'll just have to live with my users being able to send as anybody then...

24 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,789

Re: Sending mails from smtp username and aliases specified only

Wraptor wrote:

So thats a no go?

I'm afraid so.
You can try other Postfix milter program for this purpose.