From Computer Tyme Support Wiki

Revision as of 06:28, 15 March 2008

The Problem with Spam on the Internet

As Secretary General Kofi Annan said, "In its short life, the Internet has become an agent of dramatic, even revolutionary change and maybe one of today's greatest instruments of progress. It is a marvelous tool to promote and defend freedom and to give access to information and knowledge. WSIS saw the beginning of a dialogue between two different cultures: the non-governmental Internet community, with its traditions of informal, bottom-up decision-making; and the more formal, structured world of governments and intergovernmental organizations. It is my hope that the IGF will deepen this dialogue and contribute to a better understanding of how we can make full use of the potential the Internet has to offer for all people in the world."

One of the main components of this revolutionary change is email, giving people of the world free and instant communication. However, junk email is now a serious threat to the newly found communication and threatens to undermine some of the benefits of the Internet. This junk email known as "Spam" is now over 90% of all email traffic. Spam clogs the delivery of legitimate email filling email boxes and saturating communication lines so the legitimate communication is blocked. It is also a means for which people are defrauded and has a significant toll on society. The problem has risen to a level requiring that the United Nations be aware of the issue and to take steps to address the problem.

What is Spam?

Generally spam is unsolicited email where the sender has no relationship with the recipient. Generally the purpose of spam is to market products or to defraud people out of their money. Much of the volume of spam email is associated with criminal activity such as the stealing of personal identity information leading to illegal credit card transactions, stealing of money through bank accounts, impersonating financial institutions, selling of illegal merchandise, to affect stock prices in "pump and dump" scams, illegal online gambling, and other forms of crime. Because the Internet is international it has attracted a criminal element in an environment where laws can not be enforced and criminals can reach all over the world to commit crimes.

Some spam is commercial in nature where the sender is actually selling a legitimate product but is broadcasting their advertising to email lists that were bought from vendors who spider Internet web sites harvesting email addresses from random people. Because the Internet is almost free these commercial vendors can send billions of commercial messages that land in the inbox of millions of people who have no interest at all in the product that is being advertised.

There is also to a much smaller extent political spam which is also free speech. Although most spam is clearly spam there is some spam which is border line and can not be readily classified. Sometimes companies who have a relationship with a customer send those customers email advertising related products from other vendors. The email might be unsolicited or unwanted, it is not the same as if there were absolutely no relationship at all.

Another example might include peace groups who are advocating to the end of a war or occupation but are using email lists that are not targeted and the sender has no relationship to the recipient. There are people who regularly send mass mailing to the press in order to voice their opinions about issues that they consider important. Even though these are also unsolicited emails they can not be as easily classified as spam and it is important to not harm free speech in any way while trying to eradicate junk email. One must not sacrifice our liberties in favor of law enforcement. So for the purpose of this discussion, Spam is those messages of a purely commercial nature where there is no prior relationship or messages whose purpose is illegal.

The Cost of Spam

The cost of spam to society is tremendous from a lot of perspectives. Some costs are obvious, some are not. In the case of illegal activity the costs are obvious. People are tricked into giving up their personal information allowing criminals to steal their money online. I could name dozens of obvious things but I won’t. We all understand fraud.

The less obvious costs have to do with email technology. The costs of storing spam, the costs of transmitting spam, how spam block legitimate email by filling up user’s mailboxes to quota causing email they want to get to bounce. And the time it takes for the users to download spam, identify it, and delete it.

Because of spam all incoming email has to be spam filtered. This spam filtering is done automatically by spam filtering software and by hand where the email account owner has to manually determine what messages are spam and what messages are not. With the Internet email being 90% spam 9 out of 10 messages have to be deleted. This email takes a lot of resources to deal with. These resources vary depending on if the end user is in a developed country or an undeveloped country. Costs include:

Time

If a user has no spam filtering they have to spend the time and effort to hand delete 9 out of 10 emails received. This could involve hundreds of messages a day taking a significant amount of time out of the recipients daily life that could be spend doing other things like productive work or family life.

Bandwidth

With 90% of email being spam it take 10 times as much bandwidth to transfer email data, or in countries where bandwidth is limited, it takes 10 times as long to download email with spam than if there were no spam.

Storage

As spam is delivered into the user’s inbox that inbox fills up. Once the inbox is full then no more email can be added and the recipient is cut off from all new messages whether it be spam or good email. If the email provider allocates enough space to receive all this email then the vendor has to purchase disk drives big enough to hold 10 tomes as much data as would normally be needed. This drives up the costs of providing email and makes it less available to those who can’t pay for it.

Spam Filtering Costs

In order to reduce the spam problem all vendors who own the email servers now provide some kind of spam filtering software to reduce the problem. Spam filtering identifies some spam and it blocks some of it reducing the problem somewhat. In some causes it just tags messages as spam and passes them on or diverts suspected spam to a separate folder. Spam Filtering has many problems but is a necessary evil in the fight against unwanted email.

Spam filtering has an error rate and it often mistakenly blocks email that should go through. This misidentified email is referred to as a false positive. It is always a struggle for email providers to avoid false positives while providing effective and meaningful spam protection. If the filter is too aggressive it blocks good email. If the filter isn’t aggressive enough it allows too much junk email to get through which requires manual spam filtering. Even the systems that divert suspected spam to other folders are flawed in that although good email is stored there, the recipient never has the time or energy to sort through all that junk to find it. And a stored good message that is never found is no better than a deleted message.

Spam filtering costs money. It takes far more powerful servers to process email to find spam than it does to deliver email. A message that is filtered will take over 1000 times the processing power than unfiltered email. Email vendors have to bear the costs of this processing and it adds to the overall costs of email delivery. When costs are increased it burdens society and often excludes those in developing countries who can not afford to cover these increased costs.

Phishing and Identity Theft

A big problem in the email world is that it is very easy for a person to impersonate anyone or any institution. Criminals often pretend to be banks telling people that they need to update their account. But when they click on the link they are directed to a web site that looks like the back but isn’t. After the victim logs in with their username and password the criminal uses that information to log into the real bank and steal the victim’s money. Although many banks cover the theft they lose money and those losses are passed on to the rest of us in higher interest and fees.

The technology of email delivery needs to be changed so that if you get email from someone that you can be reasonably sure that it actually came from the email address that it claims to be. One such solution is to enhance the IMAP/POP protocols to include sending of email. That means that the person sending the email has the username and password to read the email of the account they claim to be. This still doesn’t guarantee that they are who they say they are. But it is a big step towards at least ensuring that the from address is from someone who can log in to the from address account

Free Speech

Spam often blocks free speech and free communication because spam crowds out good email. Spam is like weeds in a garden sucking up the resources and killing the crops through competition. Good email is blocked by spam filters, by email box quotas, by slow connections to the Internet which prevent email from being downloaded, and from lack of time and fatigue. Some people just don’t have the time so sort through thousands of junk email messages to find the good ones and even with manual filtering they delete a lot of good email.

Where Spam Comes From

Most spam comes from virus infected computers running popular commercial operating systems that have been exploited and turned into "spam zombies". The computer owner is usually unaware that their computer is under the control of criminals who control millions of exploited computers. These computers are exploited through computer viruses or worms or through software installed from web sites that trick people into installing software that contains code giving criminals remote control of the computer.

Popular commercial operating systems can be more of a problem than any other operating system for several reasons. First, commercial vendors have been slow to address the problem of viruses and does not take security as seriously as it should. When vendors do fix a problem the fix is only available to customers who have registered their products with the vendor and have set up their computers for automatic updating. To get these security updates users are forced into agreements with the vendor that many people don’t want to agree to. And there is a problem with people who are running stolen copies of commercial operating systems who are cut off from security upgrades.

Because of the huge number of exploitable computers criminal organizations have created an army of tens of millions of zombie computers that are under their control. These millions of computers are not only used for spam. They can also be used in Distributed Denial of Service (DDoS) attacks to blackmail online services or bring down critical online infrastructure cutting off millions of people from the Internet. It could even be used to crash the entire Internet itself by attacking the root name servers and bringing them all down at once. The problem of virus infected zombies is a critical issue that must be addressed and solved.

Another source of spam are web sites that are exploited through software holes in popular applications or people who’s passwords are hacked and the intruder gains access to servers and set up scripts to send spam. As hard as server owners try it is nearly impossible to keep ahead of criminals who are constantly trying to hack into and exploit servers. Many larger systems have to fend off attacks at a rate of several per minute.

Even if vendors succeed in fending off attackers from exploiting their servers the vendor has to spend a lot of time and resources to do so. This investment of time and resources takes effort and costs money adding to the total cost of providing services to the people of the planet. Every time you increase costs some people are excluded from the process.

Other spammers operate their own servers specifically to transmit spam. Many hosting companies who provide bandwidth services will shut down customers who send spam, but many vendors will provide bandwidth to anyone who is willing to pay for it regardless of the consequences to the public.

Fixing the Problem

Fixing the problem is not something that’s going to be easy. But there is a lot of things that can be done that can reduce the problem significantly. In reality that spam problem is really many problems that need to be addressed individually solving as many problems as possible in order to reduce the problem. This is a problem that is ever changing with technology and just like any kind of crime it’s a process to fight it. Spam will never be beaten, just reduced.

The problem is that the Internet was created in innocent times before anyone had any idea how big it was going to grow and how popular it would be. In that time standards have been set and it would be difficult if not impossible to change those standards to something else that is more secure, yet provides the freedom and privacy needed to protect the people of the world from being exploited from oppressive governments who would tap into email as a way of oppressing the citizens of all countries. In spite of this there are changes that could be made that will preserve or increase civil liberties while preserving compatibility with existing standards and providing an evolutionary path to a better system.

Some of the solutions involve closing loopholes that spammers use to exploit other people’s computers turning them into spam zombies. Other solutions involve changing standards so that it is more difficult to impersonate other people. There will also be a need for international laws to be enforced so that people in one nation aren’t exploiting people in other nations. The Internet is an international community so we need international solutions to deal with the problem.

Stopping Spam at the Source

There are several ways to stop spam. The most efficient blace to stop spam is at its source. To do that we have to understand where spam comes from.

Spam Zombies

The biggest sources of spam are virus infected spam zombies running on commercial operating systems. Commercial vendors have taken some steps to reduce the problem but not nearly enough steps. The problem is in part that commercial operating systems are often pirated and the pirated versions which are more exploitable are cut off from the upgrades needed to keep them from being turned into spam zombies.

Commercial vendors take the position that it has no obligation to provide services to people who steal their product, and on that point they are correct. However, pirated copies have become a toxic waste product of the commercial operating system business model and the rest of the world is not obligated to suffer extreme economic loss and loss of freedom to communicate so that commercial vendors can enrich themselves. Just like a factory can’t dump chemicals into the river and has to take responsibility for what comes from their smoke stacks, Commercial vendors have to take steps to prevent damage generated as a result of marketing their flawed operating system. They don’t have an obligation to support those who steal their product. But they do have an obligation to prevent their product from being exploited in a way that affects innocent third parties. Commercial vendors should be required to make fixes available so that their product can’t be exploited to become a hazard to the Internet.

If operating system vendors were required to make their security patches available to the public then they would reduce the amount of "toxic waste" generated as a byproduct of their marketing model. Nations should require commercial vendors to make security patches generally available in order to be able to sell windows in their country. This would be the biggest and easiest single step that could be passed to reduce not only the spam problem, but other security problems like denial of service attacks and fraud. This should be implemented immediately.

ISP Firewall Protection

In more innocent times any computer that connected to the Internet was a peer with every other computer on the net. That was a time when there were less people trying to exploit each other and most of the people in the web were sophisticated users. Now the Internet is used by people who are less technical and are more easily exploited. Times are different and end users need protections from those who would exploit them.

One of the things that can be done is that information and technology can be made available to ISPs so that they can provide firewall services to end users allowing them to surf the web while preventing the web from surfing them. However – some people want to provide web services so the end user should be able to open up services allowing them to determine what incoming traffic they will accept. End users need protection – but they also need freedom. The UN can help ISPs get the technology they need to provide both and set standards for best practices.

One trend in technology is to build Network Address Translation (NAT) into home cable modems and DSL modems. This feature that costs nothing to implement provides the home subscriber with good firewall protecting and the ability to connect multiple computers forming a home network. NAT allows the user(s) to surf the web opening connections to Internet services while blocking connections from the Internet to the users home computer. With a simple NAT firewall even if a computer has a security hole the computer is protected from intrusion because criminals are blocked from reaching the home computer.

Allowing IMAP/POP to Send Email

The email SMTP protocol was created in simpler times. One of the problems is that it is far too easy for any one person to impersonate any other person on the planet. One of the things that will reduce spam and fraud on the Internet is to make it more difficult for one person to impersonate someone what they aren’t. But to do this we need to change that way email is distributed and do it in a way that is a natural evolution of the current system.

In the beginning the Internet was a Unix network where every computer had its own SMTP server. One person would create an email that was submitted to the local SMTP server, the local server contacted the destination SMTP server and that server would deliver the message into the local email box. That method still works today but few people get their email that way.

Sender --> SMTP --> Recipient 

Today we have more of a consumer model where consumers run email clients and leave the SMTP servers to their Internet Service Providers (ISPs) The user creates an email message that is sent to their local ISP who has an SMTP server. That server accepts the email and then transfers the email by SMTP to the server that stores the incoming email for that user. Then the recipient connects to their server by POP/IMAP protocols to download their email.

Sender --> SMTP --> Sender’s ISP Server 
Sender’s ISP Server --> SMTP --> Recipient’s ISP Server
Recipient’s ISP Server --> IMAP --> Recipient

The problem is that anyone can impersonate any other person by setting their address to be anyone else on the planet. SMTP provides no checking to determine if the sender is the same person as they say they are. And the end user is using the same protocols to talk to servers that servers use to talk to each other so servers can’t tell if they are talking to legitimate servers or end users. I suggest a modification in the IMAP/POP protocols that allow for a two way transfer of email rather than requiring incoming email to be downloaded with IMAP/POP and outgoing to be SMTP.

Sender --> IMAP --> Sender’s ISP Server 
Sender’s ISP Server --> SMTP --> Recipient’s ISP Server
Recipient’s ISP Server --> IMAP --> Recipient

If IMAP and POP were enhanced to allow outgoing email to be transferred back up the same connection as incoming email it would have several advantages.

1. It would eliminate the need to configure outgoing SMTP. That makes it easier for the consumer. It would also eliminate the need for authenticated SMTP because IMAP/POP are already authenticated protocols.
   
2. Viruses would not be able to send email because the outgoing email connection, IMAP, will require a password to send email. The virus won’t have the password and won’t be able to send.
   
3. The server would accept outgoing email and label the from field to be the same as the email account preventing the user from pretending to be an email address other than the one the user authenticated as. It would then deliver the message to the local SMTP server which would then send it to the destination server.
   
4. This method allows the system to assert that the sender’s email address was sent from a person who had the ability to log in and read the email. Thus if you get an email from bill.gates@microsoft.com then you know that the person sending the email had the username and password to receive email on that account.
   
5. It would eliminate virus infected spam zombies from pretending to be SMTP servers because they would no longer be the official source of messages for domains that they pretend to be. It will be easier to create rules that keep servers from impersonating other servers when clients and servers use different protocols..
6. Protocols like SMTP-AUTH and Submission are no longer necessary. It also eliminates the problem of finding an SMTP server for outgoing email while traveling. If you can read your email under this system, you can send email.

Standardized Complaint System

One of the problems with spam is that the owners of the computers that are sending the spam and the ISP who is providing the connection to the Internet often don't have any idea that they are part of the problem. They don't know that their computers have been compromised and that they are being exploited. The people who are receiving the spam have no way without going to great effort to inform the people who can fix it that there is a problem.

In order to help curb spam and identify and inform people who have a problem that a problem exists there needs to be a system to allow those who are in a position to identify the problem to communicate with those who are in a position to fix the problem. That way computers that have been exploited can be identified early and action can be taken to correct the problem.

America Online (AOL) has developed an email system of notifications where the administrators and owners if IP block are notified by email when AOL users complain. AOL has made a step in the right direction with this notification system their system isn’t public and the notifications are not in a form that can feed into automated processes that can fix the problem automatically.

Suppose for example a criminal registered a Yahoo account and then started sending Nigerian spam indicating that a person could get millions of dollars wired into their account by replying to an email address contained within the message, With and automated notification system the end user would click on a “fraud” button and the message would instantly be sent to a process on the server hosting the user’s email. In turn the server would notify Yahoo that they were hosting an account that was receiving email to defraud people. If Yahoo were to receive 50 complaints like this within less than an hour the automated processes at Yahoo could automatically suspend the account and alert Yahoo personal that a problem has occurred. This cuts the scammer off from the victims preventing an innocent person from being defrauded.

Suppose also that the above message was sent not from Yahoo but from a virus infected zombie computer owned but some innocent person who has no idea that their computer has been hijacked by a criminal. In this example a notification would also go out to say AT&T who provides bandwidth to that customer allowing bandwidth providers to identify customers whose computers have been compromised and alerting the end user that they have a problem.

The messaging service wouldn't be an email service but some new protocol that yet needs to be written. The idea being to allow for feedback to be routed to those who need to see it while preserving the privacy of those who don't need to be exposed.

Here's an illustration of how it might work. Suppose I receive an email that is clearly a spam message. My email client will have a button that I can click to report the message as spam. That button send a message over my IMAP connection to my IMAP server which takes the message and looks up the sender information. It then sends a message to some database connected with ARIN who licenses IP space. If that database receives several other complaints from other users in a short period of time that database generated an alert message to the people who are in charge of that IP who then look into the problem where they might discover a hacked server or a virus infected customer that is spewing garbage. The messaging service would know who to deliver the message to without having to reveal private information to the sender about who controls that IP space.

This kind of alert system can stop problems in the early stages and can be used to identify compromised computers before they can do serious damage. It also preserves the privacy of the sender and the identities of all parties involved so as not to have a civil liberties impact and it doesn't require changes in international law. It allows the Internet to continue to be self governing.

In order to implement this however it would take a group of people to design the messaging system and set standards for the new protocol. It would also require that IMAP/POP protocols be expanded to allow for a messaging system so that the email recipient can communicate back to the email server information about the email. These changes would add functionality to the IMAP/POP protocol keeping current compatibility but adding new functionality.

Educating the Public

If spammers weren’t making money then spammers wouldn’t spam. This problem is money driven and it depends on naive consumers who are tricked by the messages that the spammers send. Part of the solution is to produce educational videos that would be available on the Internet from the web sites of ISPs and web based email providers. These educational videos would teach email users about different email scams and give them the knowledge to avoid being tricked. An educated public could take a huge bite out of the amount of money spammers make sending spam and reduce the incentive to send spam by making it less profitable. These videos could be produced and distributed at very low cost.

The Role of the United Nations

The Internet has been a shining light of freedom from its beginnings and those freedoms must be preserved. The last thing the world needs is to have a world wide government that locks down the Internet turning it into a tool of oppression rather than a tool of freedom. The Internet is of the people, by the people, and for the people. And in order to preserve that freedom we need to find solutions that allow the Internet to govern itself rather than to be governed by law.

To that end the United Nations can play a key roll in helping the Internet help itself. The UN can help identify problems and create and fund standards committees to create uniform systems that work together to make the web a self healing system. The United Nations also can help prevent individual nations from putting their own self interests ahead of the interests of the rest of the world. The Internet should be country neutral and person neutral so that everyone on the planet can share in the common freedom that it provides to the peoples of the world.

Solutions to problems should avoid using the law to manage the Internet if there is a technological way to solve the problem without law. Problems should be solved using free, open, and neutral standards that are designed to be effective without being intrusive. The mission of the UN would be more of a support role that an enforcement role.

Funding Projects

One role that the United Nations can help with is in funding projects that fight spam and provide internet security and educational resources to the public. Often a lone programmer or a small open source project can create a solution that would have an immense positive impact on the entire planet. If the United Nations can identify and fund such projects then for very little money the whole world can benefit.

Summary

Spam is a huge problem and it’s getting worse. It has huge costs to society and it inhibits free speech. It fills up email boxes, wastes storage, wastes bandwidth, wastes time, and it makes email slower, less reliable, and more expensive. But the problem can be solved. There are many things that can be done. Some examples include.

• Public Education Videos - Teach email users how to avoid getting ripped off online.
• Commercial Operating System Security Patches - make OS vendors release their security patches to the public.
• ISP Firewalls - Help develop standards for ISPs to provide default firewall protection to consumers.
• Enhancing IMAP/POP - Expand IMAP/POP protocols to allow the sending of email eliminating the need for end users to use SMTP for outgoing email.
• Complaint System - Implementing a standardized complaint feedback system allowing the people who see the problems to alert the people who can fix the problems.

Besides these specific recommendations there are important concepts that will lead to other solutions to the spam problem. These concepts include:

• Most spam comes from computers that are exploited through viruses, worms, security holes, or hacked.
• Protecting email users from viruses and other exploits by making patches available and developing systems like firewalls and software to protect against exploitation.
• Developing and distributing technology that prevents servers from being exploited.
• Identity Verification - Keep people from being able to easily impersonate other people or institutions.
• Feedback - Implement ways to quickly identify problems and to inform those who solve the problems.
• Education - Prevent people from being exploited by making them aware of scams.
• Money - Find ways to take the money out of spamming.

If these ideas are implemented it will take a huge bite out of the problem. The solution to spam is incremental. You have to constantly fix flaws one at a time to reduce the problem. Fighting spam is like fighting crime. It never completely goes away. You just have to keep taking steps to reduce it.

About the Author

Marc Perkel is the owner of Junk Email Filter dot com and Computer Tyme Hosting, a commercial spam filtering service and a web hosting company specializing in free speech hosting. He is also the founder of the Church of Reality, a religion based on believing in everything that is real and a religion that is online based and has declared the Internet to be a sacred service referring to it as the "Golden Router". He is also a former system administrator for the Electronic Frontier Foundation.