Hi all,
I have Jenkins job that connects remote machine via ssh and there run some commands and scripts.
I use in scripted pipeline the following:
sh “ssh -v root@$server_ip ‘cd $build_root && mkdir $build_dir’”
sh “scp -r build-scripts root@$server_ip:$build_root$build_dir”
also tried:
sh “ssh -o ‘StrictHostKeyChecking no’ root@$server_ip ‘cd $build_root && mkdir $build_dir’”
sh “scp -o StrictHostKeyChecking=no -r build-scripts root@$server_ip:$build_root$build_dir”
but I am still getting permission denied error and build fails with that.
With ssh verbose full log+debug:
10:54:10 [JNQ-Manual-Build] Running shell script
- ssh -v [email protected] ‘cd /home/prod_build/ && mkdir prod-887-implementing-prod-build-with-jenk_20181113_1053’
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Connecting to 192.168.192.17 [192.168.192.17] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr [email protected] none
debug1: kex: client->server aes128-ctr [email protected] none
debug1: kex: [email protected] need=16 dh_need=16
debug1: kex: [email protected] need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 79:98:97:ab:d3:b6:85:42:a4:fa:c0:2d:5f:a5:10:62
debug1: Host ‘192.168.192.17’ is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_0)
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_0)
or another log for other run I tried:
ssh -vvv [email protected] ‘cd /home/prod_build/ && mkdir prod-887-implementing-prod-build-with-jenk_20181112_1117’
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.192.17 [192.168.192.17] port 22.
debug1: Connection established.
debug3: Incorrect RSA1 identifier
debug3: Could not load “/var/lib/jenkins/.ssh/id_rsa” as a RSA1 public key
debug1: identity file /var/lib/jenkins/.ssh/id_rsa type 1
debug1: identity file /var/lib/jenkins/.ssh/id_rsa-cert type -1
debug3: Incorrect RSA1 identifier
debug3: Could not load “/var/lib/jenkins/.ssh/id_dsa” as a RSA1 public key
debug1: identity file /var/lib/jenkins/.ssh/id_dsa type 2
debug1: identity file /var/lib/jenkins/.ssh/id_dsa-cert type -1
debug3: Incorrect RSA1 identifier
debug3: Could not load “/var/lib/jenkins/.ssh/id_ecdsa” as a RSA1 public key
debug1: identity file /var/lib/jenkins/.ssh/id_ecdsa type 3
debug1: identity file /var/lib/jenkins/.ssh/id_ecdsa-cert type -1
debug1: identity file /var/lib/jenkins/.ssh/id_ed25519 type -1
debug1: identity file /var/lib/jenkins/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug3: load_hostkeys: loading entries for host “192.168.192.17” from file “/var/lib/jenkins/.ssh/known_hosts”
debug3: load_hostkeys: found key type ECDSA in file /var/lib/jenkins/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: [email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
…
debug2: kex_parse_kexinit: none,[email protected],zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: [email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
…
…
…
debug2: kex_parse_kexinit: none,[email protected]
debug2: kex_parse_kexinit: none,[email protected]
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: setup [email protected]
debug1: kex: server->client aes345-ctr [email protected] none
debug2: mac_setup: setup [email protected]
debug1: kex: client->server aes345-ctr [email protected] none
debug1: kex: [email protected] need=16 dh_need=16
debug1: kex: [email protected] need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 79:98:97:ab:d3:b6:87:42:a2:fa:c0:2c:5f:a5:10:62
debug3: load_hostkeys: loading entries for host “192.168.192.17” from file “/var/lib/jenkins/.ssh/known_hosts”
debug3: load_hostkeys: found key type ECDSA in file /var/lib/jenkins/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys
debug1: Host ‘192.168.192.17’ is known and matches the ECDSA host key.
debug1: Found key in /var/lib/jenkins/.ssh/known_hosts:2
debug1: ssh_ecdsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /var/lib/jenkins/.ssh/id_rsa (0x7fd0dda02940),
debug2: key: /var/lib/jenkins/.ssh/id_dsa (0x7fd0dda13660),
debug2: key: /var/lib/jenkins/.ssh/id_ecdsa (0x7fd0dda13ff0),
debug2: key: /var/lib/jenkins/.ssh/id_ed25519 ((nil)),
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_989)
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_989)
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /var/lib/jenkins/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Offering DSA public key: /var/lib/jenkins/.ssh/id_dsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Offering ECDSA public key: /var/lib/jenkins/.ssh/id_ecdsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /var/lib/jenkins/.ssh/id_ed25519
debug3: no such identity: /var/lib/jenkins/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 45 padlen 19 extra_pad 64)
debug2: we sent a password packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
Permission denied, please try again.
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 45 padlen 19 extra_pad 64)
debug2: we sent a password packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
Permission denied, please try again.
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 45 padlen 19 extra_pad 64)
debug2: we sent a password packet, wait for reply
Received disconnect from 192.168.192.17: 2: Too many authentication failures for root
=> permission denied
I order to create the ssh key I run in jenkins:
su -s /bin/bash jenkins
and then use:
cat ~/.ssh/id_rsa.pub | ssh username@remote_host “mkdir -p ~/.ssh && touch ~/.ssh/authorized_keys && chmod -R go= ~/.ssh && cat >> ~/.ssh/authorized_keys”
I took the same steps as described in https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys-on-centos7
BTW, ssh permission error sometimes appears and then build fails, but there are cases that build passes. I tried to compare logs for successful and failure and could that somehow that jenkins is trying to go to a different ssh keys location.
I don’t know how to solve this issue, any assistance will be appreciated.
Thank you:pray:,
Limor