Problems to Start G-Wan

Problems to start G-WAN

The problem is that some newer versions of hypervisors insist to report ZERO CPU, and/or ZERO CPU Core, leading to a division by zero.

Since G-WAN is optimized for multicore architectures, it queries the CPUID instruction and the OS Kernel structures to check the platform architecture and the associated OS policies (number of online and allowed CPUs).

Other web servers are not affected because they expect users to manually configure and run as many instances as desired (hereby creating the duplicated resource allocations that G-WAN was designed to avoid).

Checking both the CPUID instruction and the Kernel structure was enough, until recently. Now, for any reason, the hypervisors use broken CPUID implementations and OS Kernel structure.

This issue is affecting hosting companies (VPS servers), and Amazon EC2 instances, among others.

g-wan: building library outside of /csp, and g++ compilation problems

First, this is not a linker issue: you have "undefined symbol" rather than "unresolved symbol" as an error.

This is simply an #include issue.

1. define the main() function in your script.cpp file.

2. there's a G-WAN folder dedicated to user-defined include files called /gwan/include but you can as well use /csp/my_include.hpp... if you are using the right syntax:

For example, having #include "toto.hpp" in /csp/hello.cpp lets me reach C++ functions defined and implemented in the gwan/include/toto.hpp file (or defined in toto.hpp and implemented in a pre-compiled library linked to your script with #pragma link).

If you rather use #include <toto.hpp> then the SYSTEM INCLUDE PATH will be searched instead (and this will work providing that your library was correctly installed).

If you want to use #include "toto.hpp" for a custom folder that was not setup in the system, you can use G-WAN's #pragma include "../my_folder" directive to specify its PATH or you can explicitely specify it in each include: #include "../my_folder/toto.hpp".

Nothing fancy there, only C/C++ dependancy rules apply (and G-WAN really helps by providing alternate ways that do not involve system settings).

For libraries (see the G-WAN examples for SQLite, Cairo, mySQL, cURL, etc.) you can either use pre-installed libraries that exported their location in SYSTEM variables... or put your library in the /gwan/libraries folder and their include file in the /gwan/include folder.

When writing your own libraries, remember that they need to be pre-compiled. This means that you obviously cannot use G-WAN symbols since your compiler may #include "gwan.h" (to have the definitions) but your linker will not know from where G-WAN symbols can be found. The way around is to always use the G-WAN API from the G-WAN scripts. Your custom libraries must either be general-purpose or buffer any payload intended to be used by G-WAN. No-double copy is needed since G-WAN provides the set_reply() call to let G-WAN use persistent replies built without the reply xbuffer provided by G-WAN servlets.

Now, a last word about linking (which was not the cause of your trouble but could participate to the confusion). If you mix C and C++, use extern C {} to wrap your C++ prototypes called from C (otherwise you will really have "unresolved symbols").

With all this information, you should be ready to face every possible situation.

Bug G-WAN and Amazon EC2

The short answer is:

Newer releases of hypervisors report ZERO CPU, and/or ZERO CPU Core. This is creating a division by zero in G-WAN.

The longer story is:

As G-WAN is optimized for multicore architectures, it uses the CPUID instruction and the OS Kernel structures to find the platform architecture and the associated OS policies (the number of online/allowed CPUs).

In the past, checking the CPUID instruction and the Kernel structures worked well. But today hypervisors have switched to broken CPUID implementations and OS Kernel structures.

This problem created by hypervisors affects hosting companies (VPS servers), and Amazon EC2 instances, as well as companies and end-users.

The other web servers are not affected because users must manually configure, attach and run several server instances for each CPU Core (duplicating the resources that G-WAN uses only once).

G-wan xbuf_xcat(), is it a memory leak?

With xbuf_xcat(get_reply(argv), replycontent), RSS value grows
With xbuf_xcat(get_reply(argv), "value=1"), memory usage is stable

As shown by the working xbuf_xcat(reply, "value=1"), G-WAN automatically recycles the memory allocated for the reply xbuffer, so this is not the problem.

Your problem rather comes from how you are generating the replycontent.

And this portion of the code is missing in your question. If you are asking for assistance, it might help to show what you are doing.

---

UPDATE (following the source code disclosure in the question)

Your code is using a small buffer allocated on the stack so it does not change the memory usage of G-WAN.

Besides, you should directly write into the 'reply' xbuffer rather than into a temporary buffer which is then copied into the 'reply' xbuffer - and to make such a copy, you should rather use xbuf_cat() or xbuf_ncat(), but not xbuf_xcat().

Given this (undamaging but pointless) code, the "sudden raise" of memory you have seen could come from problems in other scripts (handlers? maintenance script?, OS/VM configuration?), or from the tests you make if you are using very high concurrencies.

Maybe you could try writing G-WAN servlets in another programming language than C (G-WAN supports 15 different programming languages, including Java, C#, Perl, Python, Ruby, etc.), this would help you to avoid most memory allocation pitfalls.

How to write complex application to be used under G-WAN

The /csp directory should only contain entry point servlets - that is, a source code file with a main() that can be invoked by a remote client.

You can have functions (called from several servlets) in source code form (or in compiled form) placed in the /includes (.h) and /libraries directories (.c, *.obj, *.a, *.lib).

Quick development and testing can be done with a servlet used to test your code until it is mature enough to be stored in /includes or /libraries.

For huge projects, it makes a lot of sense to use pre-compiled libraries as this will speed-up on-the-fly servlet compilation (as compared to a large collection of include files).

Last but not least, G-WAN being designed for lean coding, C++ might not be as optimal as plain C for this purpose - just think of the hidden cost of constructors, their order, their sometimes redundant actions, hidden memory allocations, etc. - on the top of the huge C++ runtime overhead, high compilation times, blackbox standard libraries, etc.

At TWD, for Global-WAN (relying on G-WAN), we have spent quite a lot of time at rewritting C++ (non-standard) libraries in plain C, with very tangible gains in terms of performance, bug and deadlock cleanup, and memory usage.

What you don't see might hurt - and hurt badly because, well, you don't see it until it's too late.

Hope this helps.

---

Related Topics