Troubleshooting Guide in split format
|
|
{{TableOfContents}}This page describes problems encountered using Fuego, and their solutions.
| |
|
Note for Editors: please put each issue in it's own page section | ''Note for Editors: please put each issue in it's own page section'' | |
Installation [edit section] | = Installation === Problem with default Jenkins port ==Fuego has Jenkins default to using port 8080 on the host system. However,if you have something else already running on port 8080, you may wish tochange this. | |
| To change the Jenkins port, start the container, and insidethe container edit the file: * /etc/default/jenkins | To change the Jenkins port, start the container, and insidethe container edit the file: * /etc/default/jenkins | |
| Change the line that says: HTTP_PORT=8080 | Change the line that says: HTTP_PORT=8080 | |
| Change to port to whatever your like. | Change to port to whatever your like. | |
| Also, check the line that defines JENKINS_ARGS. Mine looked like this:{{{#!YellowBoxJENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=8080 --prefix=/fuego"}}} |
Also, check the line that defines JENKINS_ARGS. Mine looked like this:{{{#!YellowBoxJENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=8080 --prefix=/fuego"}}}
| |
| Change this line to read as follows:{{{#!YellowBoxJENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --prefix=/fuego"}}} |
Change this line to read as follows:{{{#!YellowBoxJENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --prefix=/fuego"}}}
| |
| Then restart Jenkins: * $ service jenkins restart | Then restart Jenkins: * $ service jenkins restart | |
Problem creating docker file [edit section] | == Problem creating docker file ==Make sure you are running on a 64-bit version of the Linux kernel onyour host machine. | |
Problem starting Jenkins after initial container creation [edit section] | == Problem starting Jenkins after initial container creation ==Doug Crawford reported a problem starting Jenkins in the containerafter his initial build. | |
{{{#!YellowBox$ sudo ./docker-create-container.shCreated JTA container 6a420f901af7847f2afa3100d3fb3852b71bc65f92aecd13a9aefe0823d42b77$ sudo ./docker-start-container.sh Starting JTA container6a420f901af7847f2afa3100d3fb3852b71bc65f92aecd13a9aefe0823d42b77[....] Starting Jenkins Continuous Integration Server: jenkinssu: System error failed![ ok ] Starting OpenBSD Secure Shell server: sshd.[ ok ] Starting network benchmark server.}}}
| ||
| The error string is jenkinssu: System error | The error string is jenkinssu: System error | |
| Takuo Kogushi provides the following response: | Takuo Kogushi provides the following response: | |
| I had the same issue. I did some search in the net and found it is not a problem of fuego itself.As far as I know there are two workarounds; * 1) Rebuild and install libpam with --disable-audit option (in the container) or * 2) Modify docker-create-container.sh to add --pid="host" option to docker create command | I had the same issue. I did some search in the net and found it is not a problem of fuego itself.As far as I know there are two workarounds; * 1) Rebuild and install libpam with --disable-audit option (in the container) or * 2) Modify docker-create-container.sh to add --pid="host" option to docker create command | |
| Here is a patch provided by Koguchi-san:{{{#!YellowBoxdiff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-scripts/docker-create-container.shindex 2ea7961..24663d6 100755--- a/fuego-host-scripts/docker-create-container.sh+++ b/fuego-host-scripts/docker-create-container.sh@@ -7,7 +7,7 @@ while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symli done DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" |
Here is a patch provided by Koguchi-san:{{{#!YellowBoxdiff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-scripts/docker-create-container.shindex 2ea7961..24663d6 100755--- a/fuego-host-scripts/docker-create-container.sh+++ b/fuego-host-scripts/docker-create-container.sh@@ -7,7 +7,7 @@ while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symli done DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
| |
| -CONTAINER_ID=`sudo docker create -it -v $DIR/../userdata:/userdata --net="host" fuego`+CONTAINER_ID=`sudo docker create -it -v $DIR/../userdata:/userdata --pid="host" --net="host" fuego` CONTAINER_ID_FILE="$DIR/../last_fuego_container.id" echo "Created Fuego container $CONTAINER_ID" echo $CONTAINER_ID > $DIR/../last_fuego_container.id}}} | -CONTAINER_ID=`sudo docker create -it -v $DIR/../userdata:/userdata --net="host" fuego`+CONTAINER_ID=`sudo docker create -it -v $DIR/../userdata:/userdata --pid="host" --net="host" fuego` CONTAINER_ID_FILE="$DIR/../last_fuego_container.id" echo "Created Fuego container $CONTAINER_ID" echo $CONTAINER_ID > $DIR/../last_fuego_container.id}}} | |
| Actually I have not tried the first one and do not know if there is any side effects for the second.--- | Actually I have not tried the first one and do not know if there is any side effects for the second.--- | |
| This may be related to this docker bug: https://github.com/docker/docker/issues/5899 | This may be related to this docker bug: https://github.com/docker/docker/issues/5899 | |
Problem with bad port on ssh connection [edit section] | == Problem with bad port on ssh connection ==ovgen.py doesn't parse SSH_PORT from: * /home/jenkins/fuego/engine/scripts/overlays/base/base-board.fuegoclassbecause it is missing double quotes. | |
| The symptom is the following:You see the following in the test log for some test you tried to run:{{{#!YellowBox+++ sshpass -e ssh -o ServerAliveInterval=30 -oStrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -oConnectTimeout=15 -p root@10.0.0.1 trueBad port 'root@10.0.0.1'+++ abort_job 'Cannot connect to 10.0.0.1 via ssh'+++ set +}}} |
The symptom is the following:You see the following in the test log for some test you tried to run:{{{#!YellowBox+++ sshpass -e ssh -o ServerAliveInterval=30 -oStrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -oConnectTimeout=15 -p root@10.0.0.1 trueBad port 'root@10.0.0.1'+++ abort_job 'Cannot connect to 10.0.0.1 via ssh'+++ set +}}}
| |
| The error string here is "Bad port 'root@10.0.0.1'" | The error string here is "Bad port 'root@10.0.0.1'" | |
| This occurs because the port is empty. It should have been passedto the ssh command after the '-p' command line option, but since itis empty, it uses the account-name@address combination as the argument. | This occurs because the port is empty. It should have been passedto the ssh command after the '-p' command line option, but since itis empty, it uses the account-name@address combination as the argument. | |
| The reason it is empty is that a bug in the base-board.fuegoclass ismissing the double-quotes. | The reason it is empty is that a bug in the base-board.fuegoclass ismissing the double-quotes. | |
| This is fixed in the tbird20d repository with the following commit: * https://bitbucket.org/tbird20d/fuego-core/commits/abb2e7161ba66017a267c09897e5db4d938ab214 | This is fixed in the tbird20d repository with the following commit: * https://bitbucket.org/tbird20d/fuego-core/commits/abb2e7161ba66017a267c09897e5db4d938ab214 | |
General [edit section] | = General === Timeout executing ssh commands ==In some cases, the ssh command used by Fuego takes a very longtime to connect. There is a timeout for the ssh commands,specified as 15 seconds in the cogent repository and 30 secondsin the tbird20d repository. | |
| The timeout for ssh commands is specified in the file * /home/jenkins/fuego/engine/scripts/overlays/base/base-params.fuegoclass | The timeout for ssh commands is specified in the file * /home/jenkins/fuego/engine/scripts/overlays/base/base-params.fuegoclass | |
| You can change ConnectTimeout to something longer by editing the file. | You can change ConnectTimeout to something longer by editing the file. | |
| FIXTHIS - make ConnectTimeout for ssh connections a board-level test variable | FIXTHIS - make ConnectTimeout for ssh connections a board-level test variable | |
ssh commands taking a long time [edit section] | == ssh commands taking a long time ==Sometimes, even if the command does not time, the SSH operationson the target take a very long time for each operation. | |
| The symptom is that when you are watching the console output fora test, the test stops at the point of each SSH connection to thetarget. | The symptom is that when you are watching the console output fora test, the test stops at the point of each SSH connection to thetarget. | |
| One cause of long ssh connection times can be that the targetssh server (sshd) is configured to do DNS lookups on each inboundconnection. | One cause of long ssh connection times can be that the targetssh server (sshd) is configured to do DNS lookups on each inboundconnection. | |
| To turn this off, on the target, edit the file: * /etc/ssh/sshd_configand add the line: | To turn this off, on the target, edit the file: * /etc/ssh/sshd_configand add the line: | |
{{{#!YellowBoxUseDNS no}}}
| ||
| This line can be added anywhere in the file, but I recommend addingit right after the UsePrivilegeSeparation line (if that's there). | This line can be added anywhere in the file, but I recommend addingit right after the UsePrivilegeSeparation line (if that's there). |