Using a SQL Server for Application Logging. Pros/Cons

Is writing server log files to a database a good idea?

Write locally to disk, then batch insert to the database periodically (e.g. at log rollover time). Do that in a separate, low-priority process. More efficient and more robust...

(Make sure that your database log table contains a column for "which machine the log event came from" by the way - very handy!)

History tables pros, cons and gotchas - using triggers, sproc or at application level

I'd put it this way:

  • Stored procs: they're bypassed if you modify the table directly. Security on the database can control this
  • Application: same deal. Also if you have multiple applications, possibly in different languages, it needs to be implemented in each stack, which is somewhat redundant; and
  • Triggers: transparent to the application and will capture all changes. This is my preferred method.

Pros and cons of row level security Microsoft SQL Server

Pros

Row level security (RLS) reduces your application development complexity. Without RLS, you'd generally create filters with your queries or through ORM, or pass on filtering criteria to your procs. Once you have RLS in place, user1 sees what you desire user1 to see without changing your application much.

With RLS, security is at the DB level. If another application was reading data in a different language from a different platform, RLS still allows user1 to see what you want user1 to see. If someone is trying to run/test SQL queries to SSMS, RLS shows data according to your setup.

Your DB backup and restore will keep RLS intact.

Cons

The disadvantage is more DB-centric work and small amount of performance penalty. If you have web developers who aren't strong on the DB side, troubleshooting RLS might take them a bit of time. On the contrary, if application layer was filtering information through an ORM, you could debug and write auditing on the application or middle-ware side.

As far as the overheads of row level security go, here's an article that details some challenges: https://www.mssqltips.com/sqlservertip/4005/sql-server-2016-row-level-security-limitations-performance-and-troubleshooting/ and https://www.mssqltips.com/sqlservertip/4778/performance-impact-of-sql-server-2016-rowlevel-security/.

To use RLS or not?

If you are comfortable with RLS, I'd recommend you try it out. Microsoft has listed some RLS-related best practices here: https://learn.microsoft.com/en-us/sql/relational-databases/security/row-level-security?view=sql-server-2017.

If your DB is well protected and access to data is through application layer only, you can opt to build filtering within your application layer. It's not a lot of work to do security checks at the application layer AND use RLS. There isn't a one size fits all answer. For most simpler applications, I tend to use the application layer for filtering since it has served me well with auditing/logging/debugging.

What's more efficient - storing logs in sql database or files?

You can use a component such as Zend_Log which natively supports the concept of writers attached to the same log instance. In that way you can log the same message to one or more different place with no need to change your logging code. And you can always change your code to replace the log system or add a new one in a simple way.

For your question I think that log to files is simpler and more appropriate if you (developer) is the only one who needs to read log messages.

Log to db instead if you need other people needs to read logs in a web interface or if you need the ability to search through logs. As someone else has pointed out also concurrency matters, if you have a lot of users log to db could scale better.

Finally, a log frequency of 5 messages per minute requires almost no cpu for your application, so you don't need to worry about performances. In your case I'd start with logfiles and then change (or add more writers) if your requisites will change.

What are the pros and cons of performing calculations in sql vs. in your application

It depends on a lot of factors - but most crucially:

  • complexity of calculations (prefer doing complex crunching on an app-server, since that scales out; rather than a db server, which scales up)
  • volume of data (if you need to access/aggregate a lot of data, doing it at the db server will save bandwidth, and disk io if the aggregates can be done inside indexes)
  • convenience (sql is not the best language for complex work - especially not great for procedural work, but very good for set-based work; lousy error-handling, though)

As always, if you do bring the data back to the app-server, minimising the columns and rows will be to your advantage. Making sure the query is tuned and appropriately indexed will help either scenario.

Re your note:

and then loop through the records

Looping through records is almost always the wrong thing to do in sql - writing a set-based operation is preferred.

As a general rule, I prefer to keep the database's job to a minimum "store this data, fetch this data" - however, there are always examples of scenarios where an elegant query at the server can save a lot of bandwidth.

Also consider: if this is computationally expensive, can it be cached somewhere?

If you want an accurate "which is better"; code it both ways and compare it (noting that a first draft of either is likely not 100% tuned). But factor in typical usage to that: if, in reality, it is being called 5 times (separately) at once, then simulate that: don't compare just a single "1 of these vs 1 of those".

What are the pros/cons of and best practices for using a single database?

Larger database are harder to maintain due to sheer size: backups take longer, disaster recovery is slower which in turn requires more often backups. You can address these by creating filegroups and using filegroup level backup in your maintenance plans and on crash recovery you can use the 'piecemeal restore' strategy to speed things up.

Proper use of filegroups will make most of the 'cons' cited by previous replies go away: they can distribute the I/O, they can sanitize your maintenance plans and backup/restore strategy, they offer availability by taking offline only the damaged portion of the the db in case of crash. So I'd say that while those 'cons' are legit concerns, they have can be mitigated by a proper deployment strategy. Its true though that these mitigation actions require a true, experienced, dba at the helm as they will go beyond the comfort zone of a developer turned dba by need.

Some of the pros I can think of quickly:

  1. Consistency. You can have a backup-restore so that all data is consistent. Separate dbs don't allow this because you cannot coordinate a consistent set of backups unless you take them all offline, or make them r/o, during the backup.
  2. Dirt cheap high availability: you can deploy database mirroring for disaster recoverability and high availability. Multiple databases have problems because one cannot coordinate a simultaneous failover and apps are faced with the dilemma of seeking each database current location.
  3. Security. While most other posts see one database harder to secure, I'd say is easier to secure. Multiple databases seem harder to secure properly simply because what everyone does is they make one login and add it to that database db_owner group. Having one database will make things harder (unless you end up making everyone dbo, very bad) but once you start doing the right thing (granular access) then one db is not harder than multiple dbs, is actually easier because you won't have to copy/maintain some common groups/rights across multiple dbs.
  4. Control. Will be easier to impose certain policies and good practices on a single db rather than multiple ones (no data access to developers, app data access only through execute rights on the schema to enforce procedures access etc).

There are also some cons I did not see in other posts:

  1. This will be much harder to pull off that you think right now
  2. Increase coupling between formerly separated applications will impose development restrictions: you can't simply alter your schema, you will have to coordinate it with the rest of the apps (you can argue that this was also the case before, but was brushed under the carpet by having separate dbs, and you're right)
  3. Log writes that are now distributed across multiple db logs will be consolidated into one single log file. If your writes are significant, this may turn out to be a serious bottleneck and force you to buy some expensive fast drives for the new, consolidated, log file. In general this can be addresses by making the log drive a stripped array across as many stripes as needed to make it fast enough (usually raid 10).
  4. GAM/SGAM/PFS allocations will also be consolidated, but again this will be alleviated by proper use of file groups.

Pro & Cons of storing files(pictures) in a SQL Server for a website

If the files are integral and actively changing part of the system and them have to be backed up along with the other data - you can store them inside the DB, but try to use the FILESTREAM fields if you use sql server 2005+ and your files are big enough - say 500k+

If the files are static content, you can store them outside with only pointers in DB. This not prevents to take into account all your custom permissions machine.

Storing and working with files inside DB is usually slower, than in filesystem, but all depends on your needs.

Is SQL Server/Windows integrated security good for anything?

Many of these have been said or are similar to previous answers... With AD integration:

a) I don't have to worry about the users who have access to any given application, I can pass that off to the security guys.

b) I can restrict access at a table by table level based on groups that already exists, as well as forcing standard users to only have the ability to call stored proc's.

c) When a developer leaves my group, we don't have to change all the DB passwords (i.e. if you care about data security...)

d) It's easy to do custom logging based on the user who makes the change. There are other ways to do this, but I'm all about being lazy.

e) Integrates with IIS authentication. If you're already using IE & IIS for your intranet, this just makes life a lot easier.

Note: There are far more reasons not to use it, and I never used it before my present position. Here where everything is lined up in AD already... It's just the easiest time I've ever had with database security.



Related Topics



Leave a reply



Submit