Gitlab Configuration Issues:: Nginx Unicorn Port Conflict

Gitlab Configuration Issues:: NGINX Unicorn Port Conflict

You should override the ports in /etc/gitlab/gitlab.rb. Don't mess with /var/opt/gitlab/... because any manual configuration will be lost after a reconfigure. In particular read on Setting the NGINX listen port. In the downloads page there is a sentence: For troubleshooting and configuration options please see the Omnibus GitLab readme. I wonder if this is not seen by people :/ If not we should make it more clear.

Gitlab ports 80 & 8080 taken by a separate Gitlab instance?

What I have found is that there were 2 issues causing the errors I had.

First, I removed a "gitlab-ce" package that was installed, following the instructions here: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/135. For some reason, when I restart the machine now I have to restart these services, in order, for Gitlab to run properly redis-server, gitlab, nginx. However, Gitlab does start responding properly after that.

Second, the 404 error was due to a different server that was also listening on that IP address, causing a conflict.

I will likely move to using the omnibus package on a fresh, new server going forward, but at least the immediate issues appear resolved. Thanks for your help, SLY!

GitLab Unicorn web server is not running

The first line within your Unicorn logs highlights the problem:

/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:177:in `rescue in spec': Specified 'mysql2' for database adapter, but the gem is not loaded. Add `gem 'mysql2'` to your Gemfile (and ensure its version is at the minimum required by ActiveRecord). (Gem::LoadError)

You have specified that you're going to use MySQL within the config/database.yml file, but the Gem mysql2 doesn't appear to be installed. The mysql2 Gem contains the necessary libraries for interacting with your MySQL database.

The following command:

gem list

Will list the Gems that are currently installed, allowing you to verify whether mysql2 is amongst them.

This section of the installation document covers the installation of the necessary Gems for your GitLab install:

https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#install-gems

Gitlab on port 8080

Most likely you have another service listening on 8080, I think the omnibus install have some service hooking 8080 - just use 8081 instead.

Edit: I just did a quick search and found that it's the unicorn server that is listening to 8080 with the original omnibus installer.

Note: You will only need to change the external_url in gitlab.rb, no other config file should have to be edited for this.

Edit#2: As @emeraldjava stated there is an option in the configuration file for using another unicorn port: #unicorn['port'] = '8080'

Gitlab timeouts / slow on initial page loads

Jan,

I have a similar setup although our box is dedicated to GITlab. Without knowing the specs of your server (GITLAB likes memory) and the load on that box I would suggest the following diagnostics:

1. Does your upstream nginx use identical parameters as the gitlab nginx configuration? They have tweaked a number of things including timeouts.
2. What kind of request result in time outs? Some operations (like generating diffs) can take some time to render.
3. If you run the requests via SSH do you also experience time outs?
4. Have you checked global logs in /var/log?

Is it bad to use unicorn without nginx? why?

As Heroku DevCenter claims, Unicorn workers are vulnerable to slow clients.

Each worker is only able to process a single request, and if the client is not ready to accept the entire answer (aka "slow client"), the Unicorn worker is blocked on sending out the response and cannot handle the next one. Since each Unicorn worker takes up a substantial amount of RAM (again, see Heroku, it claims to handle 2-4 processes at 512 MiB RAM), you cannot rely on number of workers, since it's about the number of clients that can render your application inoperable by pretending to have slow connections.

When behind nginx, Unicorn is able to dump the entire answer into nginx's buffer and switch immediately to handling the next request.

That said, nginx with a single Unicorn worker behind is much more reliable than a bunch of Unicorn workers exposed directly.

NB: for the folks using ancient Rubies out there: if you'll be using a set of Unicorn workers, consider migrating to at least Ruby 2.0 to reduce RAM consumption by sharing common data across forked processes (ref).

Is it necessary to put Unicorn behind Nginx ( or Apache)

Unicorn was not designed to handle "slow clients". You can read more about this in the PHILOSOPHY help file:

Most benchmarks we’ve seen don’t tell you this, and unicorn doesn’t care about slow clients… but you should.

A “slow client” can be any client outside of your datacenter. Network traffic within a local network is always faster than traffic that crosses outside of it. The laws of physics do not allow otherwise.

Persistent connections were introduced in HTTP/1.1 reduce latency from connection establishment and TCP slow start. They also waste server resources when clients are idle.

Persistent connections mean one of the unicorn worker processes (depending on your application, it can be very memory hungry) would spend a significant amount of its time idle keeping the connection alive and not doing anything else. Being single-threaded and using blocking I/O, a worker cannot serve other clients while keeping a connection alive. Thus unicorn does not implement persistent connections.

If your application responses are larger than the socket buffer or if you’re handling large requests (uploads), worker processes will also be bottlenecked by the speed of the client connection. You should not allow unicorn to serve clients outside of your local network.

---

Related Topics