Forums

Resolved
0 votes
Hi,
we are having a strange issue on ClearOs 7.7 with a synology machine thats been added to the domain for years without problems, this started after these situations:

Updates on the 25 of March -Kernel and others
Updates on the 13 of May - Ldap and samba
Restarted the machine on the 21 of May for a infrastructure update, nothing related to problems in ClearOS
After the restart problems appeared with users browsing the network by VPN
Solved the problem with VPN by removing the OpenVPN app and reinstalling again
After this the authentication problem for that machine started


We already did the following, always on new machine clones:

Changed the WINADMIN account password to see if that was the problem
Started the winbind service to see if that helps, the service is not enabled on start by default
Tried to remove the computer and rejoin it again
Tried a upgrade to 7.8 but it did not solve the problem
Tried to downgrade the LDAP packages using YUM History UNDO - It does not downgrade - no packages to downgrade to
Tried to reinstall the LDAP packages using YUM History REDO - It reinstalls but the problems is still there
Tried to rollback everything with YUM History ROLLBACK - It does not downgrade LDAP packages - no packages to downgrade to
A new install of ClearOS 7 and configuration restore - the problem is still there


Yesterday while digging into the problem once more I noticed that on the LDAP values I have a SAMBA PASSWORD and a PASSWORD TIME for that machine while the ClearOS machine which is also added to the domain, of course, does not have these values. We are pulling back the old ClearOS 6 machine to check if the LDAP values mentioned are there also.

What can we see in the logs, feel free to point me to another log or so because we can be missing a spot:

--
-- /var/log/samba/optcos-optsrv02
--
[2020/08/04 17:56:06.464811, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
_netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
[2020/08/04 17:56:25.836474, 1] ../lib/param/loadparm.c:1022(lpcfg_service_ok)
NOTE: Service profiles is flagged unavailable.
[2020/08/04 17:56:25.837295, 0] ../source3/rpc_server/srv_pipe.c:1265(api_pipe_alter_context)
Auth step returned an error (NT_STATUS_WRONG_PASSWORD)
[2020/08/04 17:56:25.843309, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
_netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
[2020/08/04 17:56:25.845646, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
_netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
--
-- /var/log/messages
--
.....
Aug 4 17:59:06 optcos nmbd[1327]: [2020/08/04 17:59:06.127296, 0] ../source3/nmbd/nmbd_namequery.c:109(query_name_response)
Aug 4 17:59:06 optcos nmbd[1327]: query_name_response: Multiple (2) responses received for a query on subnet 192.168.1.8 for name OPTIMIZER<1d>.
Aug 4 17:59:06 optcos nmbd[1327]: This response was from IP 192.168.1.10, reporting an IP address of 192.168.1.10.
Aug 4 17:59:06 optcos smbd[9167]: [2020/08/04 17:59:06.623147, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:06 optcos smbd[9167]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 17:59:06 optcos smbd[9167]: [2020/08/04 17:59:06.625578, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:06 optcos smbd[9167]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 17:59:06 optcos smbd[9167]: [2020/08/04 17:59:06.635142, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:06 optcos smbd[9167]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 17:59:06 optcos smbd[9167]: [2020/08/04 17:59:06.637955, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:06 optcos smbd[9167]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 17:59:22 optcos smbd[9184]: [2020/08/04 17:59:22.392625, 0] ../source3/rpc_server/srv_pipe.c:1265(api_pipe_alter_context)
Aug 4 17:59:22 optcos smbd[9184]: Auth step returned an error (NT_STATUS_WRONG_PASSWORD)
Aug 4 17:59:22 optcos smbd[9184]: [2020/08/04 17:59:22.398617, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:22 optcos smbd[9184]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 17:59:22 optcos smbd[9184]: [2020/08/04 17:59:22.400914, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 17:59:22 optcos smbd[9184]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:05 optcos smbd[9459]: [2020/08/04 18:00:05.987020, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:05 optcos smbd[9459]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:05 optcos smbd[9459]: [2020/08/04 18:00:05.989382, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:05 optcos smbd[9459]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:06 optcos smbd[9184]: [2020/08/04 18:00:06.748202, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:06 optcos smbd[9184]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:06 optcos smbd[9184]: [2020/08/04 18:00:06.750918, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:06 optcos smbd[9184]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:10 optcos smbd[9466]: [2020/08/04 18:00:10.053741, 0] ../source3/rpc_server/srv_pipe.c:1265(api_pipe_alter_context)
Aug 4 18:00:10 optcos smbd[9466]: Auth step returned an error (NT_STATUS_WRONG_PASSWORD)
Aug 4 18:00:10 optcos smbd[9466]: [2020/08/04 18:00:10.059872, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:10 optcos smbd[9466]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:10 optcos smbd[9466]: [2020/08/04 18:00:10.062294, 0] ../source3/rpc_server/netlogon/srv_netlog_nt.c:1011(_netr_ServerAuthenticate3)
Aug 4 18:00:10 optcos smbd[9466]: _netr_ServerAuthenticate3: netlogon_creds_server_check failed. Rejecting auth request from client OPTSRV02 machine account OPTSRV02$
Aug 4 18:00:10 optcos smbd[9466]: [2020/08/04 18:00:10.881439, 0] ../source3/auth/auth.c:441(load_auth_module)
Aug 4 18:00:10 optcos smbd[9466]: load_auth_module: can't find auth method trustdomain!
Aug 4 18:00:10 optcos smbd[9466]: [2020/08/04 18:00:10.881489, 0] ../source3/auth/auth.c:434(load_auth_module)
Aug 4 18:00:10 optcos smbd[9466]: load_auth_module: auth method winbind did not correctly init
.....



The user used to join the domain in the WINADMIN account and we already changed the password to make sure that was not the problem.
On the Synology Machine we only see a message "cannot find the specified windows domain". The Synology machine has not been updated or changed in any way.

Any pointers at the moment area welcome, we are at a dead end.



Thank you.



Regards,
Alexandre.
Thursday, August 06 2020, 10:36 AM
Share this post:
Responses (8)
  • Accepted Answer

    Sunday, August 16 2020, 09:15 AM - #Permalink
    Resolved
    0 votes
    Hi Nick,
    thank you for all the help input you been giving in first place.

    For network and connectivity stability issues we can only bring down ClearOS safely on weekends to try new approaches to the problem so that's why we have been silent, sorry.
    Regarding your last tips, the Synology machine hasn't seen updates in a while because Synology is not very stable update wise... So the problem for sure was something changed on the ClearOS side or in the connectivity between them, we think that the trust password is changed from time to time by the client machine by an automated process but we were unable so far to confirm that, although the secrets.tdb previous and current password indicates that change since we dont have any way to change it our selfs.

    Good news is the problem is solved now, we were able to solve it last night and it really was a problem with the password the client thought it had with the real one stored on LDAP. Changing the password was not possible because samba interface did not allow us to write that machine entry password on secrects.tdb, or at least we did not found a suitable way to do that.

    What worked was the following, not a very nice approach but worked:

    We created a new Linux VM with the same hostname as the Synology machine
    We removed the Synology Machine from LDAP using the ClearOS web interface
    Added the new Linux VM to the domain, which generated a new entry with the same hostname.
    On that new machine a new trust password and other details were generated on the secret.tdb file as supposed.
    We then backed up and switched the original secrects.tdb on the synology machine by the new one generated on the other Linux VM.


    After those steps the domain was back Online on the Synology Machine without any issue we only had to force a refresh on the domain users list.


    Once again thank you for all of the help.


    Regards,
    Alexandre.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, August 09 2020, 12:30 PM - #Permalink
    Resolved
    0 votes
    If it does not work with both the old and new server, then it either points to the NAS being the issue or a common problem between to two ClearOS servers. The ClearOS servers use very different versions of Samba. I think 6.x uses the 3.5 line of Samba whereas 7.x is now using 4.10 and it was a huge jump from 3 to 4. 3 required SMB1 whereas 4.x supports SMB2 and SMB3 as well.

    If it is a password issue, 6.x does not have the modification to LDAP I gave you the diff for so works a smaller range of characters for the password. I think the UI validates anything, but the docs have always said not to use | ; *

    Also, if it is a password issue, change it to something simple to test.

    Could it be possible that the NAS has had an update?

    Otherwise what you are doing is beyond my experience or knowledge.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, August 09 2020, 11:53 AM - #Permalink
    Resolved
    0 votes
    Hi Nick,
    sorry about the delay but we took the weekend slowdown to test a number of stuff we came up with and you tips of course so this is what we done so far:

    We pulled back a ClearOS 6.8 machine that was working previous to the migration on March this year but the erros still continues, we also checked and the hash of the password was not the same as it is at the moment and changed it to see if that helped but no luck.
    We tried a new machine and redone the same operation that we think caused a problem wich was reinstalling OpenVPN App but no luck as well.
    We edited the UserDriver.php file as showed by you but no luck as well.
    We used tcpdump to check what was going on while trying to authenticate that machine and make sure communication was going trough
    We also raised the log level on samba to see if that said something useful


    Regarding that last step while the debug was on we got the following, it may trigger something to you that it failing us:

    [2020/08/08 17:16:30.652597, 3] ../lib/util/access.c:365(allow_access)
    Allowed connection from 192.168.1.10 (192.168.1.10)
    [2020/08/08 17:16:30.652643, 3] ../source3/smbd/service.c:603(make_connection_snum)
    make_connection_snum: Connect path is '/tmp' for service [IPC$]
    [2020/08/08 17:16:30.652677, 3] ../source3/smbd/vfs.c:113(vfs_init_default)
    Initialising default vfs hooks
    [2020/08/08 17:16:30.652692, 3] ../source3/smbd/vfs.c:139(vfs_init_custom)
    Initialising custom vfs hooks from [/[Default VFS]/]
    [2020/08/08 17:16:30.652791, 3] ../source3/smbd/service.c:849(make_connection_snum)
    optsrv02 (ipv4:192.168.1.10:46489) connect to service IPC$ initially as user guest (uid=353, gid=63000) (pid 7957)
    [2020/08/08 17:16:30.655906, 3] ../source3/rpc_server/srv_pipe.c:751(api_pipe_bind_req)
    api_pipe_bind_req: lsarpc -> lsarpc rpc service
    [2020/08/08 17:16:30.655928, 3] ../source3/rpc_server/srv_pipe.c:356(check_bind_req)
    check_bind_req for lsarpc context_id=0
    [2020/08/08 17:16:30.655941, 3] ../source3/rpc_server/srv_pipe.c:399(check_bind_req)
    check_bind_req: lsarpc -> lsarpc rpc service
    [2020/08/08 17:16:30.655973, 5] ../source3/auth/auth.c:532(make_auth3_context_for_ntlm)
    Making default auth method list for DC
    [2020/08/08 17:16:30.655985, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match anonymous
    [2020/08/08 17:16:30.655994, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method anonymous has a valid init
    [2020/08/08 17:16:30.656002, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match sam
    [2020/08/08 17:16:30.656010, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method sam has a valid init
    [2020/08/08 17:16:30.656018, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match winbind
    [2020/08/08 17:16:30.656025, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method winbind has a valid init
    [2020/08/08 17:16:30.656032, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match sam_ignoredomain
    [2020/08/08 17:16:30.656040, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method sam_ignoredomain has a valid init
    [2020/08/08 17:16:30.656079, 5] ../auth/gensec/gensec_start.c:739(gensec_start_mech)
    Starting GENSEC mechanism spnego
    [2020/08/08 17:16:30.656128, 5] ../auth/gensec/gensec_start.c:739(gensec_start_mech)
    Starting GENSEC submechanism ntlmssp
    [2020/08/08 17:16:30.656142, 3] ../auth/ntlmssp/ntlmssp_util.c:72(debug_ntlmssp_flags)
    Got NTLMSSP neg_flags=0x62088235
    NTLMSSP_NEGOTIATE_UNICODE
    NTLMSSP_REQUEST_TARGET
    NTLMSSP_NEGOTIATE_SIGN
    NTLMSSP_NEGOTIATE_SEAL
    NTLMSSP_NEGOTIATE_NTLM
    NTLMSSP_NEGOTIATE_ALWAYS_SIGN
    NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY
    NTLMSSP_NEGOTIATE_VERSION
    NTLMSSP_NEGOTIATE_128
    NTLMSSP_NEGOTIATE_KEY_EXCH
    [2020/08/08 17:16:30.657290, 3] ../source3/rpc_server/srv_pipe.c:356(check_bind_req)
    check_bind_req for lsarpc context_id=0
    [2020/08/08 17:16:30.657342, 3] ../auth/ntlmssp/ntlmssp_server.c:552(ntlmssp_server_preauth)
    Got user=[OPTSRV02$] domain=[OPTIMIZER] workstation=[OPTSRV02] len1=24 len2=24
    [2020/08/08 17:16:30.657357, 3] ../source3/param/loadparm.c:3872(lp_load_ex)
    lp_load_ex: refreshing parameters
    [2020/08/08 17:16:30.657390, 3] ../source3/param/loadparm.c:548(init_globals)
    Initialising global parameters
    [2020/08/08 17:16:30.657448, 3] ../source3/param/loadparm.c:2786(lp_do_section)
    Processing section "[global]"
    [2020/08/08 17:16:30.657875, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[optimizer_geral]"
    [2020/08/08 17:16:30.658079, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[optimizer.lan]"
    [2020/08/08 17:16:30.658140, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[homes]"
    [2020/08/08 17:16:30.658180, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[printers]"
    [2020/08/08 17:16:30.658231, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[print$]"
    [2020/08/08 17:16:30.658259, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[netlogon]"
    [2020/08/08 17:16:30.658292, 2] ../source3/param/loadparm.c:2803(lp_do_section)
    Processing section "[profiles]"
    [2020/08/08 17:16:30.658335, 1] ../lib/param/loadparm.c:1022(lpcfg_service_ok)
    NOTE: Service profiles is flagged unavailable.
    [2020/08/08 17:16:30.658350, 3] ../source3/param/loadparm.c:1621(lp_add_ipc)
    adding IPC service
    [2020/08/08 17:16:30.658380, 5] ../source3/auth/auth_util.c:122(make_user_info_map)
    Mapping user [OPTIMIZER]\[OPTSRV02$] from workstation [OPTSRV02]
    [2020/08/08 17:16:30.658391, 5] ../source3/auth/user_info.c:64(make_user_info)
    attempting to make a user_info for OPTSRV02$ (OPTSRV02$)
    [2020/08/08 17:16:30.658398, 5] ../source3/auth/user_info.c:72(make_user_info)
    making strings for OPTSRV02$'s user_info struct
    [2020/08/08 17:16:30.658407, 5] ../source3/auth/user_info.c:125(make_user_info)
    making blobs for OPTSRV02$'s user_info struct
    [2020/08/08 17:16:30.658415, 3] ../source3/auth/auth.c:189(auth_check_ntlm_password)
    check_ntlm_password: Checking password for unmapped user [OPTIMIZER]\[OPTSRV02$]@[OPTSRV02] with the new password interface
    [2020/08/08 17:16:30.658423, 3] ../source3/auth/auth.c:192(auth_check_ntlm_password)
    check_ntlm_password: mapped user is: [OPTIMIZER]\[OPTSRV02$]@[OPTSRV02]
    [2020/08/08 17:16:30.658667, 2] ../source3/passdb/pdb_ldap.c:530(init_sam_from_ldap)
    init_sam_from_ldap: Entry found for user: OPTSRV02$
    [2020/08/08 17:16:30.658811, 5] ../source3/passdb/pdb_interface.c:1748(lookup_global_sam_rid)
    lookup_global_sam_rid: looking up RID 515.
    [2020/08/08 17:16:30.659005, 4] ../source3/passdb/pdb_ldap.c:1632(ldapsam_getsampwsid)
    ldapsam_getsampwsid: Unable to locate SID [S-1-5-21-1874391424-47551290-3891210831-515] count=0
    [2020/08/08 17:16:30.659187, 2] ../source3/passdb/pdb_ldap.c:2386(init_group_from_ldap)
    init_group_from_ldap: Entry found for group: 1000515
    [2020/08/08 17:16:30.659217, 5] ../source3/passdb/pdb_interface.c:1883(pdb_default_lookup_rids)
    lookup_rids: Domain Computers:2
    [2020/08/08 17:16:30.659315, 3] ../libcli/auth/ntlm_check.c:425(ntlm_password_check)
    ntlm_password_check: NT MD4 password check failed for user OPTSRV02$
    [2020/08/08 17:16:30.659327, 5] ../source3/auth/auth.c:251(auth_check_ntlm_password)
    auth_check_ntlm_password: sam authentication for user [OPTSRV02$] FAILED with error NT_STATUS_WRONG_PASSWORD, authoritative=1
    [2020/08/08 17:16:30.659341, 2] ../source3/auth/auth.c:334(auth_check_ntlm_password)
    check_ntlm_password: Authentication for user [OPTSRV02$] -> [OPTSRV02$] FAILED with error NT_STATUS_WRONG_PASSWORD, authoritative=1
    [2020/08/08 17:16:30.659359, 2] ../auth/auth_log.c:476(log_authentication_event_human_readable)
    Auth: [lsarpc,(null)] user [OPTIMIZER]\[OPTSRV02$] at [Sat, 08 Aug 2020 17:16:30.659351 WEST] with [NTLMv1] status [NT_STATUS_WRONG_PASSWORD] workstation [OPTSRV02] remote host [ipv4:192.168.1.10:46489] mapped to [OPTIMIZER]\[OPTSRV02$]. local host [ipv4:192.168.1.8:445]
    [2020/08/08 17:16:30.659401, 2] ../lib/audit_logging/audit_logging.c:141(audit_log_json)
    JSON Authentication: {"timestamp": "2020-08-08T17:16:30.659372+0100", "type": "Authentication", "Authentication": {"version": {"major": 1, "minor": 0}, "status": "NT_STATUS_WRONG_PASSWORD", "localAddress": "ipv4:192.168.1.8:445", "remoteAddress": "ipv4:192.168.1.10:46489", "serviceDescription": "lsarpc", "authDescription": null, "clientDomain": "OPTIMIZER", "clientAccount": "OPTSRV02$", "workstation": "OPTSRV02", "becameAccount": null, "becameDomain": null, "becameSid": null, "mappedAccount": "OPTSRV02$", "mappedDomain": "OPTIMIZER", "netlogonComputer": null, "netlogonTrustAccount": null, "netlogonNegotiateFlags": "0x00000000", "netlogonSecureChannelType": 0, "netlogonTrustAccountSid": null, "passwordType": "NTLMv1", "duration": 3404}}
    [2020/08/08 17:16:30.659417, 5] ../auth/ntlmssp/ntlmssp_server.c:386(ntlmssp_server_auth_send)
    ntlmssp_server_auth_send: Checking NTLMSSP password for OPTIMIZER\OPTSRV02$ failed: NT_STATUS_WRONG_PASSWORD
    [2020/08/08 17:16:30.659431, 5] ../auth/gensec/gensec.c:492(gensec_update_done)
    gensec_update_done: ntlmssp[0x563bc0434f70]: NT_STATUS_WRONG_PASSWORD
    [2020/08/08 17:16:30.659441, 3] ../auth/gensec/spnego.c:1423(gensec_spnego_server_negTokenTarg_step)
    gensec_spnego_server_negTokenTarg_step: SPNEGO(ntlmssp) login failed: NT_STATUS_WRONG_PASSWORD
    [2020/08/08 17:16:30.659451, 5] ../auth/gensec/gensec.c:492(gensec_update_done)
    gensec_update_done: spnego[0x563bc04341a0]: NT_STATUS_WRONG_PASSWORD
    [2020/08/08 17:16:30.659471, 0] ../source3/rpc_server/srv_pipe.c:1265(api_pipe_alter_context)
    Auth step returned an error (NT_STATUS_WRONG_PASSWORD)
    [2020/08/08 17:16:30.659535, 2] ../source3/rpc_server/rpc_server.c:560(named_pipe_packet_done)
    Disconnect after fault
    [2020/08/08 17:16:30.659551, 2] ../source3/rpc_server/rpc_server.c:587(named_pipe_packet_done)
    Fatal error(Invalid argument). Terminating client(192.168.1.10) connection!
    [2020/08/08 17:16:30.663291, 3] ../source3/rpc_server/srv_pipe.c:751(api_pipe_bind_req)
    api_pipe_bind_req: netlogon -> netlogon rpc service
    [2020/08/08 17:16:30.663313, 3] ../source3/rpc_server/srv_pipe.c:356(check_bind_req)
    check_bind_req for netlogon context_id=0
    [2020/08/08 17:16:30.663326, 3] ../source3/rpc_server/srv_pipe.c:399(check_bind_req)
    check_bind_req: netlogon -> netlogon rpc service
    [2020/08/08 17:16:30.663355, 5] ../source3/auth/auth.c:532(make_auth3_context_for_ntlm)
    Making default auth method list for DC
    [2020/08/08 17:16:30.663366, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match anonymous
    [2020/08/08 17:16:30.663375, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method anonymous has a valid init
    [2020/08/08 17:16:30.663383, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match sam
    [2020/08/08 17:16:30.663391, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method sam has a valid init
    [2020/08/08 17:16:30.663398, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match winbind
    [2020/08/08 17:16:30.663406, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method winbind has a valid init
    [2020/08/08 17:16:30.663413, 5] ../source3/auth/auth.c:412(load_auth_module)
    load_auth_module: Attempting to find an auth method to match sam_ignoredomain
    [2020/08/08 17:16:30.663424, 5] ../source3/auth/auth.c:437(load_auth_module)
    load_auth_module: auth method sam_ignoredomain has a valid init
    [2020/08/08 17:16:30.663463, 5] ../auth/gensec/gensec_start.c:739(gensec_start_mech)
    Starting GENSEC mechanism schannel
    [2020/08/08 17:16:30.663541, 3] ../auth/gensec/schannel.c:618(schannel_update_internal)
    Could not find session key for attempted schannel connection from OPTSRV02: NT_STATUS_NOT_FOUND
    [2020/08/08 17:16:30.663558, 5] ../auth/gensec/gensec.c:492(gensec_update_done)
    gensec_update_done: schannel[0x563bc042b390]: NT_STATUS_NOT_FOUND
    [2020/08/08 17:16:30.663577, 2] ../source3/rpc_server/srv_pipe.c:570(pipe_auth_generic_bind)
    ../source3/rpc_server/srv_pipe.c:570: auth_generic_server_step[68/6] failed: NT_STATUS_NOT_FOUND
    [2020/08/08 17:16:30.663602, 2] ../source3/rpc_server/rpc_server.c:560(named_pipe_packet_done)
    Disconnect after fault
    [2020/08/08 17:16:30.663611, 2] ../source3/rpc_server/rpc_server.c:587(named_pipe_packet_done)
    Fatal error(Invalid argument). Terminating client(192.168.1.10) connection!
    [2020/08/08 17:16:30.676106, 3] ../source3/smbd/service.c:1129(close_cnum)
    optsrv02 (ipv4:192.168.1.10:46489) closed connection to service IPC$
    [2020/08/08 17:16:30.676164, 2] ../source3/smbd/utmp.c:439(sys_utmp_update)
    utmp_update: uname:/var/run/utmp wname:/var/log/wtmp
    [2020/08/08 17:16:30.676374, 3] ../source3/smbd/server_exit.c:237(exit_server_common)
    Server exit (NT_STATUS_END_OF_FILE)


    At the moment we are pursuing this options side by side:
    1 - We used tdbdump to check the secrect.tdb on the client machine (Synology NAS) and check the passwords for the trust relation
    2 - We are using Kali to reverse engineer the HASH on Ldap sambaNTPassword and be sure of what is that password.

    With the first option we found that the file secrets stores 2 values for the trust password, the current and the previous:

    root@OPTSRV02:/etc/samba/private/bckopt# tdbdump secrets.tdb.
    ....
    {
    key(39) = "SECRETS/MACHINE_PASSWORD.PREV/OPTIMIZER"
    data(15) = "Le#Mp5=NrMQF[b\00"
    }
    {
    key(34) = "SECRETS/MACHINE_PASSWORD/OPTIMIZER"
    data(15) = "OI15uSeL%8bK_]\00"
    }
    ....


    We then tried authentication with those password and later on we change a users ldap password with those password to check and verify the generated hash.
    Authentication using the previous and the current password, on that order:

    root@OPTSRV02:/etc/samba/private/bckopt# wbinfo -a optimizer\\OPTSRV02$
    Enter optimizer\OPTSRV02$'s password:
    plaintext password authentication failed
    Could not authenticate user optimizer\OPTSRV02$ with plaintext password
    Enter optimizer\OPTSRV02$'s password:
    challenge/response password authentication succeeded
    root@OPTSRV02:/etc/samba/private/bckopt# wbinfo -a optimizer\\OPTSRV02$
    Enter optimizer\OPTSRV02$'s password:
    plaintext password authentication failed
    Could not authenticate user optimizer\OPTSRV02$ with plaintext password
    Enter optimizer\OPTSRV02$'s password:
    challenge/response password authentication failed
    wbcAuthenticateUserEx(optimizer\OPTSRV02$): error code was NT_STATUS_WRONG_PASSWORD (0xc000006a)
    error message was: Wrong Password
    Could not authenticate user optimizer\OPTSRV02$ with challenge/response


    You can see that the previous password is the current one on LDAP, we later checked it to be true using the hash by, like I said, changing a users password to that one and the hash is the same. So following that route we tried putting in the current password in a user to get the generated hash correct and see if that solved our problem, so we did but it started returning a another error:

    root@OPTSRV02:/etc/samba/private/bckopt# wbinfo -a optimizer\\OPTSRV02$
    Enter optimizer\OPTSRV02$'s password:
    plaintext password authentication failed
    Could not authenticate user optimizer\OPTSRV02$ with plaintext password
    Enter optimizer\OPTSRV02$'s password:
    challenge/response password authentication failed
    wbcAuthenticateUserEx(optimizer\OPTSRV02$): error code was NT_STATUS_ACCESS_DENIED (0xc0000022)
    error message was: Access denied
    Could not authenticate user optimizer\OPTSRV02$ with challenge/response


    So as a wild guess we are assuming the "%" character in the current password is possibly messing up things.. is this a possibility??

    We now are trying a way to change the secrets.tdb current password for something else in order to test it, but wbinfo -c does not alow us to change the password without an active domain connection...

    This afternoon we will be spining a new clone to check you tip on SMB protocol version, and after I you let you know how it went.



    Thank you a lot for the input you been giving on this matter.



    Regards,
    Alexandre.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, August 09 2020, 07:56 AM - #Permalink
    Resolved
    0 votes
    I've just seen a post on the samba mailing list. Can you try this, but it will mess up the rest of your network when you try it. In /etc/samba/smb.conf set:
    server max protocol = NT1
    domewhere in the global section. Then restart smb. If it works and you need, it you would then need to enable SMB1 on all your Win10 machines.

    If it works, please can you report back?
    The reply is currently minimized Show
  • Accepted Answer

    Friday, August 07 2020, 08:25 AM - #Permalink
    Resolved
    0 votes
    It is hard to go back with some packages and impossible with others. You can't go back with any app- packages unless they are still in the base "clearos" repo which dates back to 7.7. You would need to try a "yum downgrade app-package app-package-core". If it then fails on a dependency (unlikely) you'll have to downgrade the dependency at the same time, but youm should show you what to do. In clearos-updates there is only ever one version of the package. For other packages such as samba which are in the clearos-centos* and clearos-epel repos it is possible to go back, but you will need to track down the dependencies. For example:
    [root@server ~]# yum downgrade samba --assumeno
    Loaded plugins: clearcenter-marketplace, fastestmirror
    ClearCenter Marketplace: fetching repositories...
    Loading mirror speeds from cached hostfile
    * clearos: mirror1-frankfurt.clearos.com
    * clearos-centos-sclo-rh: download4.clearsdn.com
    * clearos-centos-verified: mirror1-frankfurt.clearos.com
    * clearos-contribs: mirror1-frankfurt.clearos.com
    * clearos-contribs-paid: mirror1-frankfurt.clearos.com
    * clearos-epel-verified: mirror1-frankfurt.clearos.com
    * clearos-fast-updates: download4.clearsdn.com
    * clearos-infra: mirror1-frankfurt.clearos.com
    * clearos-paid: mirror1-frankfurt.clearos.com
    * clearos-verified: mirror1-frankfurt.clearos.com
    * private-clearcenter-antimalware: download2.clearsdn.com:80
    * private-clearcenter-antispam: download1.clearsdn.com:80
    * private-clearcenter-business: download1.clearsdn.com:80
    * private-clearcenter-content-filter: download2.clearsdn.com:80
    * private-clearcenter-dnsthingy: download1.clearsdn.com:80
    * private-clearcenter-ids: download4.clearsdn.com:80
    * private-clearcenter-master-slave: download3.clearsdn.com:80
    * private-clearcenter-nextcloud-business: download1.clearsdn.com:80
    * private-clearcenter-rbs: download1.clearsdn.com:80
    * private-clearcenter-roundcubemail: download4.clearsdn.com:80
    * private-clearcenter-security-audit: download4.clearsdn.com:80
    * private-clearcenter-smart-monitor: download3.clearsdn.com:80
    * private-clearcenter-verified-updates: download2.clearsdn.com:80
    Resolving Dependencies
    --> Running transaction check
    ---> Package samba.x86_64 0:4.10.4-10.el7 will be a downgrade
    --> Processing Dependency: libwbclient = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-client-libs = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-common = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-common = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-common-libs = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-common-tools = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    --> Processing Dependency: samba-libs = 4.10.4-10.el7 for package: samba-4.10.4-10.el7.x86_64
    ---> Package samba.x86_64 0:4.10.4-11.el7_8 will be erased
    --> Finished Dependency Resolution
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: libwbclient = 4.10.4-10.el7
    Installed: libwbclient-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    libwbclient = 4.10.4-11.el7_8
    Available: libwbclient-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    libwbclient = 4.9.1-6.el7
    Available: libwbclient-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    libwbclient = 4.9.1-10.el7_7
    Available: libwbclient-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    libwbclient = 4.10.4-10.el7
    Error: Package: samba-dc-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    Requires: samba = 4.10.4-11.el7_8
    Removing: samba-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba = 4.10.4-11.el7_8
    Downgraded By: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba = 4.10.4-10.el7
    Available: samba-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba = 4.9.1-6.el7
    Available: samba-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba = 4.9.1-10.el7_7
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: samba-libs = 4.10.4-10.el7
    Installed: samba-libs-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba-libs = 4.10.4-11.el7_8
    Available: samba-libs-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba-libs = 4.9.1-6.el7
    Available: samba-libs-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba-libs = 4.9.1-10.el7_7
    Available: samba-libs-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba-libs = 4.10.4-10.el7
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: samba-common-tools = 4.10.4-10.el7
    Installed: samba-common-tools-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba-common-tools = 4.10.4-11.el7_8
    Available: samba-common-tools-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba-common-tools = 4.9.1-6.el7
    Available: samba-common-tools-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba-common-tools = 4.9.1-10.el7_7
    Available: samba-common-tools-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba-common-tools = 4.10.4-10.el7
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: samba-client-libs = 4.10.4-10.el7
    Installed: samba-client-libs-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba-client-libs = 4.10.4-11.el7_8
    Available: samba-client-libs-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba-client-libs = 4.9.1-6.el7
    Available: samba-client-libs-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba-client-libs = 4.9.1-10.el7_7
    Available: samba-client-libs-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba-client-libs = 4.10.4-10.el7
    Error: Package: samba-python-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    Requires: samba = 4.10.4-11.el7_8
    Removing: samba-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba = 4.10.4-11.el7_8
    Downgraded By: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba = 4.10.4-10.el7
    Available: samba-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba = 4.9.1-6.el7
    Available: samba-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba = 4.9.1-10.el7_7
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: samba-common = 4.10.4-10.el7
    Installed: samba-common-4.10.4-11.el7_8.noarch (@clearos-centos-verified)
    samba-common = 4.10.4-11.el7_8
    Available: samba-common-4.9.1-6.el7.noarch (clearos-centos-verified)
    samba-common = 4.9.1-6.el7
    Available: samba-common-4.9.1-10.el7_7.noarch (clearos-centos-verified)
    samba-common = 4.9.1-10.el7_7
    Available: samba-common-4.10.4-10.el7.noarch (clearos-centos-verified)
    samba-common = 4.10.4-10.el7
    Error: Package: samba-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    Requires: samba-common-libs = 4.10.4-10.el7
    Installed: samba-common-libs-4.10.4-11.el7_8.x86_64 (@clearos-centos-verified)
    samba-common-libs = 4.10.4-11.el7_8
    Available: samba-common-libs-4.9.1-6.el7.x86_64 (clearos-centos-verified)
    samba-common-libs = 4.9.1-6.el7
    Available: samba-common-libs-4.9.1-10.el7_7.x86_64 (clearos-centos-verified)
    samba-common-libs = 4.9.1-10.el7_7
    Available: samba-common-libs-4.10.4-10.el7.x86_64 (clearos-centos-verified)
    samba-common-libs = 4.10.4-10.el7
    You could try using --skip-broken to work around the problem
    You could try running: rpm -Va --nofiles --nodigest
    These dependencies may then reveal other dependencies. You may also have to specify the version number you want to downgrade to is you don't just want the prior one.

    I am not aware of any update to LDAP unless you are doing manual updates rather than automatic. Openldap was last October and it was a trivial update to try to delay winbind from starting until samba openldap had started. App-openldap-directory was updated more recently and that again was a minor update to allow extra characters in the password. The diff is:
    [devel@microserver libraries]$ git diff 5dbada902371e51590fc2254195c66c7838b8655 User_Driver.php
    diff --git a/libraries/User_Driver.php b/libraries/User_Driver.php
    index d7aad22..dcd931a 100644
    --- a/libraries/User_Driver.php
    +++ b/libraries/User_Driver.php
    @@ -654,12 +654,12 @@ class User_Driver extends User_Engine

    $shell = new Shell();
    $intval = $shell->execute(
    - self::COMMAND_LDAPPASSWD,
    - '-x ' .
    - '-D "' . $dn . '" ' .
    - '-w "' . $oldpassword . '" ' .
    - '-s "' . $password . '" ' .
    - '"' . $dn . '"',
    + self::COMMAND_LDAPPASSWD,
    + "-x " .
    + "-D '" . $dn . "' " .
    + "-w '" . $oldpassword . "' " .
    + "-s '" . $password . "' " .
    + "'" . $dn . "'",
    FALSE, $options
    );
    and it would be easy to revert manually. As it affects passwords, you could try it. App-openldap has not updated for even longer.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, August 07 2020, 02:19 AM - #Permalink
    Resolved
    0 votes
    Hi,
    sorry about the delay but been reading and trying a number of things to try and get to the source of the problem.

    We have enabled the Winbind service as you said and tried your solution but we still have the same error.
    Like I said before we checked the hash on the sambaNTPassword on an old ClearOS 6 and the current one and tried reverting the value since it was different but the problem persists.

    From the client machine if we do a
    wbinfo -t
    it returns with sucess stating the domain is online, but the problem persists stating
    NT_STATUS_WRONG_PASSWORD
    .


    Any more ideas?
    Is there anyway I can install a new machine and stop it from updating to these package versions we now have and restore a configuration backup?
    We have migrated to ClearOS in March and at that point all was working smoothly.


    Regards,
    Alexandre.
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, August 06 2020, 12:52 PM - #Permalink
    Resolved
    0 votes
    Hi Nick,
    thanks for the help in first place.

    We will be checking your tip later (in about 5 hours) after the majority of people left and we only need to stop remote workers from logging in. We will clone the current instance and make changes on the clone first.
    The samba share is not a problem, all of our shares are working great up until now even with the Syn machine unable to find the domain, what tipped us for the problem was that new users are not being synced by Synology so if we want some special permission on a share at the moment are unable to do it for newly added users, up until that point in time (about a month an half ago ) all users are cached so it still works!!

    Meanwhile, like I said we checked the Hash on sambaNTPassword on an old ClearOs 6 and the active one 7.7 and they differ, and no update was done on this, so we will also be trying to change the value directly on LDAP and check if that solves our problem.


    Thanks once again, will post in a few hours.


    Regards,
    Alexandre.
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, August 06 2020, 11:12 AM - #Permalink
    Resolved
    0 votes
    Winbind should be enabled to start on boot and must be running with the latest version of Samba. It may be worth stopping all samba:
    systemctl stop smb nmb winbind
    Then removing /var/lib/samba/wins.*, /var/lib/samba/gencache.tdb and /var/lib/samba/winbindd_cache.tdb then restarting samba.

    I know a long time ago when mapping a Pi to use a samba share in its line in /etc/fstab I had to make sure I had the parameters "domain=home,vers=3.0" in the line. Obviously adjust the domain to match yours, but note this is not for a domain joined device, it is for simple file sharing.
    The reply is currently minimized Show
Your Reply