How I Should Run My Golang Process in Background

How I should run my Golang process in background?

As Nick mentioned Supervisord is a great option that has also worked well in my experience.

Nick mentioned problems with forking- forking itself works fine AFAICT. The issue is not forking but dropping privileges. Due to the way the Go runtime starts the thread pool that goroutines are multiplexed over (when GOMAXPROX > 1), the setuid systemcall is not a reliable way to drop permissions.

Instead, you should run your program as a non-privileged user and use the setcap utility to grant it the needed permissions.

For example, to allow binding to a low port number (like 80) run will need to run setcap once on the executable: sudo setcap 'cap_net_bind_service=+ep' /opt/yourGoBinary

You may need to install setcap: sudo aptitude install libcap2-bin

Can i run a golang application as a background process? ( without nohup)

For those who are interested. i solved this using the github.com/kardianos/service package. It worked by installing the application as a service and I can start or stop it from the command line without blocking. Tested out and it worked on both Linux and Windows, which served my purpose.

Sample code will me like such :-

package main

import (
    "flag"
    "fmt"
    "log"
    "net/http"

    "github.com/kardianos/service"
)

var logger service.Logger
    
// Program structures.
//  Define Start and Stop methods.
type program struct {
    exit chan struct{}
}

func (p *program) Start(s service.Service) error {
    if service.Interactive() {
        logger.Info("Running in terminal.")
    } else {
        logger.Info("Running under service manager.")
    }
    p.exit = make(chan struct{})

    // Start should not block. Do the actual work async.
    go p.run()
    return nil
}
func (p *program) run() error {
    logger.Infof("I'm running %v.", service.Platform())
    http.HandleFunc("/", handler)
    log.Fatal(http.ListenAndServe(":8080", nil))
    return nil
}
func (p *program) Stop(s service.Service) error {
    // Any work in Stop should be quick, usually a few seconds at most.
    logger.Info("I'm Stopping!")
    close(p.exit)
    return nil
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
    svcFlag := flag.String("service", "", "Control the system service.")
    flag.Parse()

    options := make(service.KeyValue)
    options["Restart"] = "on-success"
    options["SuccessExitStatus"] = "1 2 8 SIGKILL"
    svcConfig := &service.Config{
        Name:         "GoExample",
        DisplayName:  "Go Example",
        Description:  "This is a Go application.",
        Dependencies: []string{},
        // "Requires=network.target",
        // "After=network-online.target syslog.target"
        Option: options,
    }

    prg := &program{}
    s, err := service.New(prg, svcConfig)
    if err != nil {
        log.Fatal(err)
    }
    errs := make(chan error, 5)
    logger, err = s.Logger(errs)
    if err != nil {
        log.Fatal(err)
    }

    go func() {
        for {
            err := <-errs
            if err != nil {
                log.Print(err)
            }
        }
    }()

    if len(*svcFlag) != 0 {
        err := service.Control(s, *svcFlag)
        if err != nil {
            log.Printf("Valid actions: %q\n", service.ControlAction)
            log.Fatal(err)
        }

        return
    }

    err = s.Run()
    if err != nil {
        logger.Error(err)
    }

}

How do I get my Golang web server to run in the background?

Simple / Usable things first

If you want a start script without much effort (i.e. dealing with the process, just having it managed by the system), you could create a systemd service. See Greg's answer for a detailled description on how to do that.
Afterwards you can start the service with

systemctl start myserver

Previously I would have recommended trying xinetd or something similar for finer granuarlity regarding resource and permission management but systemd already covers that.

Using the shell

You could start your process like this:

nohup ./myexecutable &

The & tells the shell to start the command in the background, keeping it in the job list.
On some shells, the job is killed if the parent shell exits using the HANGUP signal.
To prevent this, you can launch your command using the nohup command, which discards the HANGUP signal.

However, this does not work, if the called process reconnects the HANGUP signal.

To be really sure, you need to remove the process from the shell's joblist.
For two well known shells this can be achieved as follows:

bash:

./myexecutable &
disown <pid>

zsh:

./myexecutable &!

Killing your background job

Normally, the shell prints the PID of the process, which then can be killed using the kill command, to stop the server. If your shell does not print the PID, you can get it using

echo $!

directly after execution. This prints the PID of the forked process.

go run app.go always run in the background

Did you tried running your server using nohup?

nohup go run server.go &

golang exec background process and get its pid

You don't need anything special, just don't tell ssh to background itself and don't Wait() for it. Example:

$ cat script.sh
#!/bin/sh
sleep 1
echo "I'm the script with pid $$"
for i in 1 2 3; do
        sleep 1
        echo "Still running $$"
done
$ cat proc.go
package main

import (
       "log"
       "os"
       "os/exec"
)

func main() {
     cmd := exec.Command("./script.sh")
     cmd.Stdout = os.Stdout
     err := cmd.Start()
     if err != nil {
        log.Fatal(err)
     }
     log.Printf("Just ran subprocess %d, exiting\n", cmd.Process.Pid)
}
$ go run proc.go
2016/09/15 17:01:03 Just ran subprocess 3794, exiting
$ I'm the script with pid 3794
Still running 3794
Still running 3794
Still running 3794

Keep running Go Server as background process

We have to inspect an always-on server from 2 perspectives:

1. Handle / tolerate errors that occur during serving requests
2. Restart the server app if it crashes or gets killed

For the first, you don't have to do anything special. If your handler panics, it will not blow your whole server, the http server will recover from that. It will only stop serving that specific request. Of course you can create your own handler which calls other handlers and recovers on panic and handle it in an intelligent way, but this is not a requirement.

^{One thing to note here: a Go app ends when the main goroutine ends (that is: the main() function returns). So even though goroutines serving requests are protected, if your main goroutine would end (e.g.
panic), your app would still exit.}

For the second, it's not really Go related. For example if you're on linux, simply install / register your compiled Go executable as a service, properly configured to have it restarted should it exit or crash.

For example in Ubuntu which uses systemd for service configuration, the following minimal service descriptor would fulfill your wishes:

[Unit]
Description=My Always-on Service

[Service]
Restart=always
Type=simple
ExecStart=/path/to/your/app -some=params passed

[Install]
WantedBy=multi-user.target

Put the above text in a file e.g. /etc/systemd/system/myservice.service, and you can enable and start it with:

sudo systemctl enable myservice.service 
sudo systemctl start myservice.service 

The first command places a symlink to the proper folder to have it auto-started on system startup. The second command starts it right now.

To verify if it's running, type:

sudo systemctl status myservice.service 

(You can omit the .service extension in most cases.)

Now whenever your app crashes, or the OS gets restarted, your app will be automatically started / restarted.

Further reading and tutorials for systemd:

How To Use Systemctl to Manage Systemd Services and Units

Systemd Essentials: Working with Services, Units, and the Journal

Convert "run at startup" script from upstart to systemd for Ubuntu 16

systemd: Writing and Enabling a Service

Execute command in background

While in general you need goroutines to run something parallel (or more precisely concurrent), in case of running an external command or app in that manner does not require you to use goroutines (in fact, it's redundant).

This is because the exec.Cmd used to run commands has a Cmd.Start() method which starts the specified command but does not wait for it to complete. So you are free to do other stuff while it runs in the background, and when you need to wait for it to finish (and process its result), you may call Cmd.Wait() (which will block and wait for the command to complete).

This is how it could look like:

cmd := exec.Command("npm", "install", "other_params")
cmd.Dir = entryPath

if err := cmd.Start(); err != nil {
    log.Printf("Failed to start cmd: %v", err)
    return
}

// Do other stuff while cmd runs in background:
log.Println("Doing other stuff...")

// And when you need to wait for the command to finish:
if err := cmd.Wait(); err != nil {
    log.Printf("Cmd returned error: %v", err)
}

In contrast to Cmd.Start(), there is Cmd.Run() which starts the specified command and waits for it to complete, should you not need to run it in the "background". In fact Cmd.Run() is nothing more than a chaining of Cmd.Start() and Cmd.Wait() calls.

Note that when running in the "background", to get the output of the app, you can't call Cmd.Output() or Cmd.CombinedOutput() as they run then command and get its output (and you already started the command). If you need the output of the command, set a buffer to Cmd.Stdout which you can inspect / use after.

This is how it can be done:

cmd := exec.Command("npm", "install", "other_params")
cmd.Dir = entryPath
buf := &bytes.Buffer{}
cmd.Stdout = buf

if err := cmd.Start(); err != nil {
    log.Printf("Failed to start cmd: %v", err)
    return
}

// Do other stuff while cmd runs in background:
log.Println("Doing other stuff...")

// And when you need to wait for the command to finish:
if err := cmd.Wait(); err != nil {
    log.Printf("Cmd returned error: %v", err)
    // You may decide to continue or return here...
}

fmt.Println("[OUTPUT:]", buf.String())

If you also want to capture the standard error stream of the app, you may / have to do the same with Cmd.Stderr. Tip: you may set the same buffer to Cmd.Stdout and Cmd.Stderr, and then you'll get the combined output, this is guaranteed as per doc:

If Stdout and Stderr are the same writer, and have a type that can be compared with ==,

at most one goroutine at a time will call Write.

What's the best practices to run a background task along with server listening

Yes, and remember that main is a goroutine too, here is the working code:

package main

import (
    "fmt"
    "net/http"
    "time"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func checkExpire() {
    for {
        // do some job
        fmt.Println(time.Now().UTC())
        time.Sleep(1000 * time.Millisecond)
    }
}

func main() {
    go checkExpire()
    http.HandleFunc("/", handler) // http://127.0.0.1:8080/Go
    http.ListenAndServe(":8080", nil)
}

Run the code and open your browser.

---

And never use Empty loop (for{}) see:

Difference between the main goroutine and spawned goroutines of a Go program

Empty loop uses 100% of a CPU Core, to wait for some operation depending to the use case you may use:

- sync.WaitGroup like this

- select {} like this

- channels

- time.Sleep

---

Related Topics