Plug-In Architecture Based C/C++ Application

plug-in architecture based c/c++ application

I think, this is not the answer you expect, but you could try to examine the Rainmeter sources. It's written in C++ (some places could be done better, to my mind, but overall it's ok) and the whole application is done the way so it just handles plugins.

Even the simple API is done via plugins, there is also a bunch of examples of contributed plugins, I mean, written by someone else (I did that too, one day).

I think you could actually study a lot of new tricks in the plugin-based development by looking at other applications.

Also to mention, another good sample is Miranda IM.

Edit: Also, if I han the same task, I would actually add some python (or something like that) backend to my application and use it as the language for SDK (for example, using boost::python).

How To Create a Flexible Plug-In Architecture?

This is not an answer as much as a bunch of potentially useful remarks/examples.

One effective way to make your application extensible is to expose its internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof (if your primitives are well chosen and implemented). A success story of this kind of thing is Emacs. I prefer this to the eclipse style plugin system because if I want to extend functionality, I don't have to learn the API and write/compile a separate plugin. I can write a 3 line snippet in the current buffer itself, evaluate it and use it. Very smooth learning curve and very pleasing results.
One application which I've extended a little is Trac. It has a component architecture which in this situation means that tasks are delegated to modules that advertise extension points. You can then implement other components which would fit into these points and change the flow. It's a little like Kalkie's suggestion above.
Another one that's good is py.test. It follows the "best API is no API" philosophy and relies purely on hooks being called at every level. You can override these hooks in files/functions named according to a convention and alter the behaviour. You can see the list of plugins on the site to see how quickly/easily they can be implemented.

A few general points.

Try to keep your non-extensible/non-user-modifiable core as small as possible. Delegate everything you can to a higher layer so that the extensibility increases. Less stuff to correct in the core then in case of bad choices.
Related to the above point is that you shouldn't make too many decisions about the direction of your project at the outset. Implement the smallest needed subset and then start writing plugins.
If you are embedding a scripting language, make sure it's a full one in which you can write general programs and not a toy language just for your application.
Reduce boilerplate as much as you can. Don't bother with subclassing, complex APIs, plugin registration and stuff like that. Try to keep it simple so that it's easy and not just possible to extend. This will let your plugin API be used more and will encourage end users to write plugins. Not just plugin developers. py.test does this well. Eclipse as far as I know, does not.

Plug-in architecture, access to code in application?

On OS X there are two types of loadable things, a dylib and a plugin. (These two terms have specialized technical meaning in the context of mach-o, the binary format OS X uses.)

A loaded dylib can't refer to the libraries in the executable, while a loaded plugin can. As a side effect, a dylib can be loaded to any executable, but a plugin can only be loaded into the executable you specify when you make the plugin.

So you want to make a plugin. There is a template in the XCode to do that. Don't forget to specify the target executable in the linker flag, which can be set somewhere in the inspector.

For more, read Code Loading Programming Topics.

Best way to implement plugin framework - are DLLs the only way (C/C++ project)?

Do not worry about malicious plugins. If somebody managed to sneak a malicious DLL into that folder, they probably also have the power to execute stuff directly.
As an alternative to DLLs, you could hook up a scripting language like Python or Lua, and allow scripted plugins. But maybe in this case you need the speed of compiled code?
For embedding Python, see here. The process is not very difficult. You can link statically to the interpreter, so users won't need to install Python on their system. However, any non-builtin modules will need to be shipped with your application.
However, if the language does not matter much to you, embedding Lua is probably easier because it was specifically designed for that task. See this section of its manual.
See 1. They don't.
Using a plugin model sounds like a fine solution, provided that a lack of extensibility really is a problem at this point. It might be easier to hard-code your current model, and add the plugin interface later, if it turns out that there is actually a demand for it. It is easy to add, but hard to remove once people started using it.

How to design extensible software (plugin architecture)?

IF we're talking .NET, try Scripting .NET applications with VBScript over on CodeProject. Lots of concrete examples there.

Below are sites implementing various application extension techniques

ClearScript - Makes V8, VBScript and JScript available to .NET apps
CS-Script - The C# Script Engine
Plugin Architecture using C#
Opinio plugin architecture
Notes on the Eclipse Plug-in Architecture
Plug-in Architecture Framework for Beginners
Gecko plugin architecture
Fungimol plugin architecture

how to develop plug-in based application?

If you're using Qt, you may find its support for plugins useful.

http://qt-project.org/doc/qt-5.0/qtcore/plugins-howto.html
http://qt-project.org/doc/qt-5.0/qtcore/qpluginloader.html

That said, this is nowhere near what you get with the Eclipse RCP framework.

If your plug-ins are released on a different schedule than the host application, compatibility among versions is an issue. Maintaining binary compatibility between minor versions, as Qt itself has typically done, reduces the concern.

Testing supported version combinations in-house is desirable before they are tested by customers.

What are the advantages and disadvantages of plug-in based architecture?

The benefits of a pluggable system are

extensibility: the application can be dynamically extended to include new features.
parallel development: since features can be implemented as separate components, they can be developed in parallel by different teams.
clear development direction: since the plugin framework ideally provides a well-defined interface and documentation for plugin writers, developers have a clear roadmap for development.
simplicity: a plugin typically has one function, and so developers have a single focus

But some of these strengths are also weaknesses:

extensibility: does the plugin interface anticipate the ways plugin writers what to extend the app, or does it restrict extension. Designing extensibility to meet all use cases often takes several iterations, or extremely good requirements analysis.
maintainability: the provider of the plugin framework not only has to make sure the plugin interface satisfies indented use cases, is clear and well documented, but also that it can evolve. Managing versions and backwards compatibility with existing plugins can be very hard. Hard enough that many practical implementations don't bother, and push the onus on plugin writers to update their plugins with each version.
complexity: although each plugin works when tested alone, interactions between plugins can cause new problems, with bugs appearing only with certain combinations of plugins.
testing: testing plugins can be difficult if the plugin system does not provide some form of mock plugin runner for testing, which is sometimes not possible, and testing is only available by running the plugin for real, which slows down development.
artifical separation: a plugin typically has a single focus, but what constitues a single focus is set by the plugin api provider. If a plugin writer finds he needs a plugin that can reasonably do 2 things (as defined by the plugin api) in close tandem, he may end up having to implement two plugins and find ways of providing communication between them that is not presently provided by the api. He's then having to work around or against the plugin framework.

Designing a good plugin environment has many of the same challenges as designing a good library. If you are producing both the environment and the plugins yourself, then it's not so bad since you can update all the plugins as the environment evolves, but if the plugin api is open to all, then it requires careful planning and execution to get the design right to avoid too many plugin rewrites as the environment evolves.

"Second-system syndrome" described by Fred Brooks advocates that the second system developed is often excessively generic, aiming for ultimate flexibility, sometimes producing a "platform within a platform"/"inner platform effect". A pluggable design is often seen as a way out when requirements are non-existent or underspecified. To compensate, the software is made as flexible as possible to try to handle "whatever comes along".

Appologies if this paints a dreary picture - pluggable systems can be fantastic and offer a lot of strengths, but they come at a high price. Before diving into a pluggable system, it's prudent to draw up requirements for all the plugins that you will need to cover the functionality required. This will then help you decide if the pluggable design is worth the effort, or some simpler approach would serve equally well.

Plug-In Architecture Based C/C++ Application