“I need to nail this board down. Got a hammer?” asked Jim, the lowly construction worker.

John, the foreman replied, “We have a pneumatic nail driver we’ve been using. You’ll have to wait for that one.”

“But it’s just for a temporary brace while I put this wall up,” said Jim.

“We only do nailing with a pneumatic nail driver on this project.”

“It’s in too tight of a space to even fit the nair with this wall here. I can just reach a hammer in and nail it down,” said Jim.

John said definitively “No, we have to use the pneumatic nail driver. Everyone knows that hammers aren’t as good. You’ll need to cut a hole in that wall so it will fit in there.”

Any seasoned developer or any CS graduate can give you a laundry list of best practices when developing software. Things like “Don’t use globals,” “Have explicit garbage collection instead of depending on built in cleanup,” or “The use of GOTO will cause searing pain.” Good developers, however, will know that there are times when deviations to these rules is acceptable, and will at times produce software that doesn’t follow every best practice.

There are a couple of barriers to always following every best practice 100% of the time – developers who work for money are usually required to produce something functional on a fairly regular basis, and sometimes breaking a “best practice” provides a performance gain that cannot be easily translated following a “best practice.”

Now, this certainly isn’t to say that going cowboy/rogue and throwing out coding standards is ideal. Breaking these rules should by far be an exception, but there are times when it can be beneficial. If bending or breaking a rule gets you significant gain without impacting performance, security, maintainability, or portability of any likely foreseeable use of the code, I say go ahead and break it.

PHP has always been a “get it done” language.  Rasmus created the language as a tool to allow him to get things done faster.  The language certainly has changed since its original inception, but the “get it done” spirit of the language has always been fostered.  Sure, there are pieces of the language that don’t make a lot of sense, like  underscores or no underscores between words in standard functions, needle haystack precedence, etc.  Yet it is still arguably the most commonly used language to develop websites.  And that’s because with all of its quirks and peculiarities, it gets the job done.

Joel Spolsky refers to people who do this as “duct tape” programmers.  They may not have every aspect of the software done with every layer of abstraction that your CS book says you need.  But they deliver software that is maintainable, done close to on schedule, and most importantly, works.

So follow the best practices wherever you can.  But when it makes sense to deviate from your CS Best Practices book, remember you’re coding to deliver a working product, not to get a grade on your source code.

Salt is good; but if the salt becomes unsalty, with what will you make it salty again?

- Jesus

Salting your hashes is a good thing.  It adds another level of protection to your hash, and prevents the effectiveness of rainbow tables should your hash get out.  It’s been standard practice to use a salt when storing passwords and similar information for almost 30 years now.  Using a weak salt can significantly reduce the protection it will give you.  Also, use of a salt should just be an additional level of protection to your hashes – not your only protection.  However, the use of a salt doesn’t suddenly mean that sharing your hashes with the world is a good thing.

Not too long ago, I was reviewing the code on a smaller open source CMS. I came across this block of code

function setUserForeverCookie() {
$hashVal = md5(PASSWORD_SALT . $this->getUserID());
setcookie("ccmUserHash", $this->getUserID() . ':' . $hashVal, time() + 1209600, DIR_REL . '/');
}

What function does is create a forever login cookie value out of an md5 hash of the combination of your user id and a static PASSWORD_SALT value.  The PASSWORD_SALT value is a number between 100,000 and 999,999, created upon the installation of the CMS.  The user id is a publicly availble id used to identify the user throughout the site, such as in the URL to the user’s profile page.  For the default super admin, the user id is always 1.

By creating an account and getting your own forever login value, you could easily discover this weak salt.
for ($i = 100000; $myForeverLogin != $guess; $i++) {
$guess = md5($i.$myUserId);
}

echo "Salt is $i\n"
echo "Admin cookie is ".md5($i.1);


In seconds those few lines of code will give you the server salt and the admin’s forever login cookie, giving you full control of the site.

While the use of something like a forever login cookie inherently poses some security risk and is prohibited from use in such regulated industries as financial or healthcare (you always need to login when you visit your bank, for example), there are times that it’ll come up as a requirement and it might make sense to do with your web app.

A better implementation of a forever login would combine a stronger salt with a number used once, or nonce, which would be stored in the database with a user record. A stronger salt would be something like a 32 character key of random letters & numbers. The hash algorithm probably wouldn’t be md5 either, and instead use a stronger encryption algorithm. By using both a strong salt stored outside of the database and a nonce in the database, a breach of either the code or the database alone would not allow a user to create a forever login.

The Internet offers the potential for safe, convenient new ways to shop for financial services and conduct banking business, any day, any time.

-FDIC, Safe Internet Banking

Anytime you visit a website, you’re providing the oppurtunity for that website to discover a lot of information about yourself.  Cross-site scripting (XSS) vulnerabilities in websites could allow a malicious 3rd party site to gain access to your email, your Facebook account, your bank account, credit cards, etc.  Security vulnerabilities in browsers, link handlers, embedded media players, and other software could be used to compromise your local machine and monitor everything you do in the background.

Even if all the security vulnerabilites across the Internet were somehow fixed, the conveniences browsers offer you by design still create the potential for misues and abuse.  All of the main browsers save some sort of browsing history.  They also will provide the ability for a website you visit to change the color of visited links.  By creating a page that has a list of links, and then iterating through them to see what color they are, you can find out what sites that computer visits.

Your browsing history is likely to show whether you use MySpace or Facebook, Google or Yahoo, which bank you use, what sites you shop at, what YouTube videos you’ve watched, etc.  This information could be used for your benefit, like linking an address to your preferred mapping website.  It could also be used for marketing research, or to get enough information to make a good social engineering request to get even more information.

This isn’t considered a security vulnerability, it’s part of the design of the Internet.  As such, the use of anti-spyware and anti-virus software will do nothing to protect you.  There are no files you have to install, no popus you have to allow or deny.  You unknowingly provide all of this information by simply viewing a web page.

To protect yourself from this, one of the best options is to eiter turn off or clear your browser history regularly.  With the many browser security vulnerabilities that exist, even better practice is to use a separate browser than what you normally use to visit sites that contain sensitive information, like your bank.

For a quick demonstration of this concept in action, visit http://www.whoismybank.com.  This site does not store any of your information.

References:

http://jeremiahgrossman.blogspot.com/2006/08/i-know-where-youve-been.html

http://www.cs.indiana.edu/~sstamm/projects/recon/

Get several more uses from your toothpaste by placing the tube in boiling water for several minutes to loosen up the toothpaste inside, then pulling the tube down across the edge of the bathroom counter to scrape all of the toothpaste from the bottom of the tube to the top.

Once this technique no longer yields more toothpaste from the tube, use a pair of scissors to cut the tube open about an inch or two from the top. Open the top part and use your toothbrush to scrape the toothpaste out of the inside of the tube.

-Associated Content

When it comes to improving performance in a PHP system, there are a few different approaches you can take.  First, find out where your system performance can best be improved – see PHP Optimization – Where to Start.  Once you know where your biggest bottlenecks are, you can try to optimize through them, but at some point, you’ll reach the limits of what optimization can do for you –  run out of toothpaste, so to speak.  There are two widely used performance techniques beyond optimization – eliminate it, or scale it out.

Optimize

Generally speaking, optimizing is getting the system to do the same thing, only faster. Optimization techniques include things like tuning the server configuration for optimal performance, refactoring the code to get better performance from PHP, Opcode caching or adding indexes to the database tables for faster reads. Optimizing is usually one of the first tools you’ll be pulling out of your bag to deal with performance issues.  But there are limits as to how far optimization will take you.

With optimization, there is the law of diminishing returns.  Your first pass at optimization may yield a 600%-800% performance improvement.  Maybe your next one will give you a 200% improvement.  The next one may still give you a 100% improvement.  With each pass, finding ways to get those big performance gains will become increasingly difficult.  At some point, you’ll be struggling to get a 5% performance boost.  Or maybe you’ll find a way to increase performance, but it will also mean the code will become unmaintainable (Even if your one supergenius coder on the team can work with it, if it’s so confusing no other programmer can, it’s unmaintainable.  More on that later).

At some point, it’s not going to be worth trying to continue down this route as your performance needs increase.  I don’t boil my toothpaste and split it open to get every last drop, I go buy a new tube of toothpaste.  Similarly, at some point, even if I know that there are still probably some performance improvements somewhere still be in the system, I’ll start looking for something other than optimization to improve system performance.  This is where elimination and scaling come into play.

Eliminate

Elimination is getting rid of your bottlenecks by finding a way to not have to execute them.  There are several different ways to do this.  Caching and using background process are commonly used ones.  Caching is probably the most common elimination technique – if there is data that does not need to be updated with every run, saving and reusing the processed information will save you from having to process it each time.  

Another common elimination technique is to take the heavy lifting off of the PHP loader and put it into separate running background process.  If there is a “heavy” process that doesn’t need to be real time, just close to real time, maybe it would make sense to have a cron job do the processing of this information.  The web user would then just look at this pre-processed information instead of processing every time.  This works two ways too – If a user uploads data, instead of waiting for it to be processed in real-time, maybe you could just hold it in a queue and tell them that it will be processed later.

Scale

Even with using optimization and elimination, or if you need to deal with real-time processes where elimination isn’t an option, you’ll reach some limits and need to be able to scale your PHP system to get better performance.  There’s “vertical scaling,” which means doing something like buying a 48-core server with a terrabyte of RAM; kind of an expensive route.  And then there’s horizontal scaling, which is setting up your system so that it can be split across multiple pieces of hardware to distribute the load.  This same technique can also be used to give you highly available redundant systems.

Designing a system to horizontally scale typically requires the most work and has the most complexity of any of these three routes. And for many PHP systems, it’s a necessary route to take somewhere in the system.  Deciding what needs to be able to horizontally scale is extremely situationally dependent.  And with the added complexity to the development and the extra development time and costs that comes with this, designing to scale in situations that don’t need it is a very poor use of resources.

Summing it up

When it comes to improving PHP performance, there are many approaches that you can take, with no blanket solutions that will solve your problems in every situation.  With each option, there are pros and cons to consider that will be impacted by many factors, anything from the deployment to the makeup of team that’s working on it.  Having a big bag of tools to approach performance issues is necessary, especially for those times you can’t rely on being able to boil out a little more toothpaste.

http://xkcd.com/327/

It’s a well known, well documented, and well abused fact that SQL injection attacks can take place in the WHERE clause of a SQL statement. The commonly applied practice among professionals is to run user input through mysql(i)_real_escape_string(). However, this only protects against user variables within quoted values, and does not protect against SQL injection attacks elsewhere in the query.

One place that is commonly vulnerable is in the ORDER BY clause. Many developers either do not understand that mysql(i)_real_escape_string does not protect them from these types of attacks, or do not think that meaningful SQL injection can be done at this point in the query on a single statement engine like MySQL. As a result, this vulnerability can be found and exploited in many applications and websites, both commercial and open source, personal and corporate.

Vulnerable code and SQL queries is basically:

<php
 
$sortColumn = mysqli_real_escape_string($_GET['sort_column']);
$query ="SELECT * from some_table WHERE active = true ORDER BY $sortColumn DESC";
 
?>

This is vulnerable to a SQL injection attack that will allow a hacker to get information from any table in the database, whether it’s usernames, passwords, credit card account numbers, etc.
 

How this can be exploited
 

The core theory behind the exploit is that this vulnerable query allow you to test a tiny piece of information from anywhere in the database in a boolean query that doesn’t rely on any unescaped characters, then use the value of that boolean to visibly change the output of the query.

Assume that the vulnerable site is a news site and lets you sort the article listings by the date or title column.  When you click on the column header you want to sort by, it sends a ’sort_column’ parameter to the above script of either ‘date’ or ‘title’.  

If instead of sending ‘date’ or ‘title’, you sent something like the following string, you would be able to start reading information from anywhere in the database.  In this particular case, we’ll try the users table.

(CASE WHEN (SELECT ASCII(SUBSTRING(password, 1, 1)) FROM users where username = 0×61646D696E) = 65 THEN date ELSE title END)

Assuming that this is the correct table and column names, this injection will allow you to tell whether or not the first character of the admin user’s password is ‘A’. If it is, the article list will be returned sorted by date.  If not, it will be returned sorted by title.

If it isn’t a match, then the 65 in the query just needs to be incremented/decremented until the match is made to try other various letters/symbols. Once the match is made and the first character is discovered, the substring offset just needs to be incremented to get the second character, but this time starting with null to see if the end of the string has already been reached. If not, start back at 65, and repeat the process until null matches.

This does require some knowledge about the database schema, which can be guessed, looked up on open source applications, or can be learned by first querying against a known table like the information schema.

A script can be written to do automate this process very quickly, as an 8 character password with upper and lowercase letters and numbers can be discovered with a maximum of 500 queries. MD5 encoded passwords will have the hashes revealed in less than 512 queries, which can then be brute force decoded (at over 500 million attempts/second, thanks to GPU computing), or directly looked up if the password is a common word or phrase.
 

Why This Works

Because each of these queries puts user input in a place in the query that is not enclosed with ‘, there is no need to use any of the characters that would be escaped by mysql(i)_real_escape_string(). Instead, SQL can be directly passed directly into the query. In places that strings are normally used when making a query, Hex notation, ASCII or other character conversion can be used to convert strings to or from their numeric values. As demonstrated in these examples, anywhere that SQL can be injected into a query is a major security vulnerability.
 

How to Secure

Securing this type of query is a rather simple process.  If a column name is expected, the user input should be validated against a whitelist array. 

Applying this on the example query:

<php
$columns = array(
'title',
'date'
);
if (in_array($_GET['sort_column'], $columns)) {
$sortColumn = $_GET['sort_column'];
} else {
$sortColumn = 'title';
}

?>

As you can see, the above code will ensure that only expected/allowed values make it through to the database. So remember, trust no one, and sanitize everything, regardless of how harmless you may think invalid input will be.

“Our web app is slow,” complained the boss.
“Well, I hear that using commas instead of periods with echo() makes PHP faster,” replied the code monkey.
“Well, let’s try optimizing that,” said the boss.

$100k later.
“Well, it’s still slow after that change. I guess we’ve reached the limits of PHP. Time to switch to Java.”

If you run a website that becomes successful, at some point, you’re going to run into performance issues. Heck, you may only need a single visitor with some scripts to bring it to a crawl. There are a few different approaches often used for optimizing PHP, but really only one right one. First there’s the old school way, dump in a bunch of var_dumps of script execution time and hope to find a slow spot. Not very efficient.

Then there’s what I like to call the cluster-bomb optimizing approach. These are those lists of 40 things to do to optimize PHP and the like. I call this the cluster-bomb approach because you’re just throwing a bunch of tiny optimizations all over the place hoping that you hit one that’s causing your performance issues. A lot of these recommendations will leave you with a trashed collection of unmaintainable code. A cluster-f*** of code so to speak.

Very rarely will the majority of these recommendations actually make an impact to a real-world performance issue. I know this because I regularly fix real-world performance issues, and I rarely follow most of the optimization “tips” in those lists.

And then there’s the profiling method. This I think is the only right way to begin any sort of optimization process, in PHP or any other language. For the uninitiated, a profiler is a process that will sit with your script handler and generate a detailed log of exactly what happened during a script run, how long a subprocess took, etc. You can then take this log and get a visual representation of your scripts performance. From there, it’s really easy to see where the performance bottlenecks are in the system, and give you a good idea of where the best effort could be put in optimization.

For PHP optimization and profiling, the tools that I like to use are Xdebug and KCacheGrind. Xdebug is the profiler, and KCacheGrind is the graphical reader of the profiler. First, install Xdebug on your PHP server. You’ll need to be able to generate a profiler log. You can do this by either enabling it in your core php.ini file, activating it from within your script, or, my preference, turning on the xdebug.profiler_enable_trigger setting in your php.ini so you just have to add XDEBUG_PROFILE=1 as a parameter in your URL or POST parameters, and you can generate the profiler files easily on demand.

Once you’ve generated the profiler log, open it up in KCacheGrind. This is a very powerful tool that I won’t be covering very in depth, but it does make finding problem spots in the code very simple. Using the Callee Map tab on the right side of the screen, you will see a visual representation of how long the calls in your script were for each run. The larger the box, the more time relative to the other process was spent in that process. By clicking on a box, you can see a stack trace of where that specific box is.

The visual profile of a complex OO script may look something like this

KCacheGrind Dump

The largest block there represents 17% of the run time of that script. If I wanted to improve performance of this script, I may want to look there. Or maybe this tells me that I need to optimize system performance by looking at something other than script internals. Or maybe I need to look at ways to reduce the number of calls in the script run.  Each situation has its own unique requirements, and there is often more than one good approach to solving the problem.

If you have a slow performing script with a bottleneck, you may see something more like this.

KCachegrind dump of bottlenecked PHP script

The two largest boxes there represent 80% of the total script run time. This is definitely where the script bottleneck is, and performance efforts would be best directed. Directing any effort anywhere else wouldn’t have any significant impact as it represents <20% of the script run.  In this particular case, the two boxes were a couple of sloppy SQL queries (something, btw, which isn’t even on the 40 ways to optimize PHP list). But by cleaning those up, I was able to easily do a 70% performance improvement to the script run time.

Now, a code bottleneck is certainly not going to be the cause of every problem, but it will be found with the majority of performance issue you’ll come across. And each bottleneck needs to be treated differently, more to come on that later. But by using the right tools and taking the few minutes to analyze your script, you’ll quickly gain excellent insight into how to best direct your optimization efforts.

Using these tools to find where to optimize, I was able to reduce script run time by 70% in about 20 minutes. Let’s see you do that by swapping out your double quotes for single quotes (#28).