File-sharing Goes Social
The RIAA has taken us on a tour of networking strategies in the last few years, by constantly changing the environment file-sharing systems operate in. In hostile environments, organisms often adapt to become less energetic but harder to kill, and so it is now. With the RIAA's waves of legal attacks driving experimentation with decentralized file-sharing tools, file-sharing networks have progressively traded efficiency for resistance to legal attack.
The RIAA has slowly altered the environment so that relatively efficient systems like Napster were killed, opening up a niche for more decentralized systems like Gnutella and Kazaa. With their current campaign against Kazaa in full swing, we are about to see another shift in network design, one that will have file sharers adopting tools originally designed for secure collaboration in a corporate setting.
Napster's problem, of course, was that although Napster nodes acted as both client and server, the central database still gave the RIAA a single target. Seeing this, Gnutella and Kazaa shifted to a mesh of nodes that could each act as client, server, and router. These networks are self-assembling and self-reconfiguring with a minimum of bootstrapping, and decentralize even addresses and pointers to files.
The RIAA is now attacking these networks using a strategy that could be called Crush the Connectors. A number of recent books on networks, such as Gladwell's The Tipping Point, Barabasi's Linked, and Watts' Six Degrees, have noted that large, loosely connected networks derive their effectiveness from a small number of highly connected nodes, a pattern called a Small World network. As a result, random attacks, even massive ones, typically leave the network only modestly damaged.
The flipside is that attacks that specifically target the most connected nodes are disproportionately effective. The RIAA's Crush the Connectors strategy will work, not simply because highly publicized legal action will deter some users, but because the value of the system will decay badly if the RIAA succeeds in removing even a small number of the best-provisioned nodes.
However, it will not work as well as the RIAA wants, even ignoring the public relations fallout, for two reasons. The first is that combining client, server, and router in one piece of software is not the last move available to network designers -- there is still the firewall. And the second is simply the math of popular music -- there are more people than songs.
Networks, Horizons, and Membranes
Napster was the last file-sharing system that was boundary-less by design. There was, at least in theory, one Napster universe at any given moment, and it was globally searchable. Gnutella, Kazaa, and other similar systems set out to decentralize even the address and search functions. This made these systems more robust in the face of legal challenges, but added an internal limit -- the search horizon.
Since such systems have no central database, they relay requests through the system from one node to the next. However, the "Ask two friends to ask two friends ad infinitum" search method can swamp the system. As a result, these systems usually limit the spread of search requests, creating an internal horizon. The tradeoff here is between the value of any given search (deeper searches are more effective) vs the load on the system as a whole (shallower searches reduce communications overhead.) In a world where the RIAA's attack mode was to go after central resources, this tradeoff worked well -- efficient enough, and resistant to Napster-style lawsuits.
However, these systems are themselves vulnerable in two ways -- first, anything that reduces the number of songs inside any given user's search horizon reduces the value of the system, causing some users to defect, which weakens the system still further. Second, because search horizons are only perceptual borders, the activity of the whole network can be observed by a determined attacker running multiple nodes as observation points. The RIAA is relying on both weaknesses in its current attack.
By working to remove those users who make a large number of files persistently available, the RIAA can limit the amount of accessible music and the trust the average user has in the system. Many of the early reports on the Crush the Connectors strategy suggest that users are not just angry with the RIAA, but with Kazaa as well, for failing to protect them.
The very fact that Crush the Connectors is an attack on trustworthiness, however, points to one obvious reaction: move from a system with search horizons to one with real membranes, and making those membranes social as well as technological.
Trust as a Border
There are several activities that are both illegal and popular, and these suffer from what economists call high transaction costs. Buying marijuana involves considerably more work than buying roses, in part because every transaction involves risk for both parties, and in part because neither party can rely on the courts for redress from unfair transactions. As a result, the market for marijuana today (or NYC tattoo artists in the 1980s, or gin in the 1920s, etc) involves trusted intermediaries who broker introductions.
These intermediaries act as a kind of social Visa system; in the same way a credit card issuer has a relationship with both buyer and seller, and an incentive to see that transactions go well, an introducer in an illegal transaction has an incentive to make sure that neither side defects from the transaction. And all parties, of course, have an incentive to avoid detection.
This is a different kind of border than a search horizon. Instead of being able to search for resources a certain topological distance from you, you search for resources a certain social distance from you. (This is also the guiding principle behind services like LinkedIn and Friendster, though in practice they represent their user's networks as being much larger than real-world social boundaries are.)
Such a system would add a firewall of sorts to the client, server, and router functions of existing systems, and that firewall would serve two separate but related needs. It would make the shared space inaccessible to new users without some sort of invitation from existing users, and it would likewise make all activity inside the space unobservable to the outside world.
Though the press is calling such systems "darknets" and intimating that they are the work of some sort of internet underground, those two requirements -- controlled membership and encrypted file transfer -- actually describe business needs better than consumer needs.
There are many ways to move to such membrane-bounded systems, of course, including retrofitting existing networks to allow sub-groups with controlled membership (possibly using email white-list or IM buddy-list tools); adopting any of the current peer-to-peer tools designed for secure collaboration (e.g. Groove, Shinkuro, WASTE etc); or even going to physical distribution. As Andrew Odlyzko has pointed out, sending disks through the mail can move enough bits in a 24 hour period to qualify as broadband, and there are now file-sharing networks whose members simply snail mail one another mountable drives of music.
A critical factor here is the social fabric -- as designers of secure networks know, protecting the perimeter of a network only works if the people inside the perimeter are trustworthy. New entrants can only be let into such a system if they are somehow vetted or vouched for, and the existing members must have something at stake in the behavior of the new arrivals.
The disadvantage of social sharing is simple -- limited membership means fewer files. The advantage is equally simple -- a socially bounded system is more effective than nothing, and safer than Kazaa.
If Kazaa, Gnutella and others are severely damaged by the Crush the Connectors attack, users will either give up free file-sharing, or switch to less efficient social spaces. This might seem like an unalloyed win for the RIAA, but for one inconvenient fact: there are more people than are songs.
There Are More People Than Songs
For the sake of round numbers, assume there are 500 million people using the internet today, and that much of the world's demand for popular music would be satisfied by the availability of something like 5 million individual songs (Apple's iTunes, by way of comparison, is a twentieth of that size.) Because people outnumber songs, if every user had one MP3 each, there would be a average of a hundred copies of every song somewhere online. A more realistic accounting would assume that at least 10% of the online population had at least 10 MP3 files each, numbers that are both underestimates, given the popularity of both ripping and sharing music.
Worse for the RIAA, the popularity of songs is wildly unequal. Some songs -- The Real Slim Shady, Come Away With Me -- exist on millions of hard drives around the world. As we've moved from more efficient systems like Napster to less efficient ones like Kazaa, it has become considerably harder to find bluegrass, folk, or madrigals, but not that much harder to find songs by Britney, 50 Cent, or John Mayer. And as with the shift from Napster to Kazaa, the shift from Kazaa to socially-bounded systems will have the least significant effect on the most popular music.
The worst news of all, though, is that songs are not randomly distributed. Instead, user clusters are a good predictor of shared taste. Make two lists, one of your favorite people and another of your favorite songs. What percentage of those songs could you copy from those people?
Both of those lists are probably in the dozens at most, and if music were randomly distributed, getting even a few of your favorite songs from your nearest and dearest would be a rare occurrence. As it is, though, you could probably get a significant percentage of your favorite songs from your favorite people. Systems that rely on small groups of users known to one another, trading files among themselves, will be less efficient than Kazaa or Napster, but far more efficient than a random distribution of music would suggest.
What Happens Next?
Small amounts of social file-sharing, by sending files as email attachments or uploading them to personal web servers, have always co-existed with the purpose-built file-sharing networks, but the two patterns may fuse as a result of the Crush the Connectors strategy. If that transition happens on a large scale, what might the future look like?
Most file-sharing would go on in groups from a half dozen to a few dozen -- small enough that every member can know every other member by reputation. Most file-sharing would take place in the sorts of encrypted workspaces designed for business but adapted for this sort of social activity. Some users would be members of more than one space, thus linking several cells of users. The system would be far less densely interconnected than Kazaa or Gnutella are today, but would be more tightly connected than a simple set of social cells operating in isolation.
It's not clear whether this would be good news or bad news for the RIAA. There are obviously several reasons to think it might be bad news: file-sharing would take place in spaces that would be much harder to inspect or penetrate; the lowered efficiency would also mean fewer high-yield targets for legal action; and the use of tools by groups that knew one another might make prosecution more difficult, because copyright law has often indemnified some types of non-commercial sharing among friends (e.g. the Audio Home Recording Act of 1992).
There is also good news that could come from such social sharing systems, however. Reduced efficiency might send many users into online stores, and users seeking the hot new song might be willing to buy them online rather than wait for the files to arrive through social diffusion, which would effectively turn at least some of these groups into buyers clubs.
The RIAA's reaction to such social sharing will be unpredictable. They have little incentive to seek solutions that don't try to make digital files behave like physical objects. They may therefore reason that they have little to lose by attacking social sharing systems with a vengeance. Whatever their reaction, however, it is clear that the current environment favors the development and adoption of social and collaborative tools, which will go on to have effects well outside the domain of file-sharing, because once a tool is adopted for one purpose, it often takes on a life of its own, as its users press such social tools to new uses.
The RIAA has slowly altered the environment so that relatively efficient systems like Napster were killed, opening up a niche for more decentralized systems like Gnutella and Kazaa. With their current campaign against Kazaa in full swing, we are about to see another shift in network design, one that will have file sharers adopting tools originally designed for secure collaboration in a corporate setting.
Napster's problem, of course, was that although Napster nodes acted as both client and server, the central database still gave the RIAA a single target. Seeing this, Gnutella and Kazaa shifted to a mesh of nodes that could each act as client, server, and router. These networks are self-assembling and self-reconfiguring with a minimum of bootstrapping, and decentralize even addresses and pointers to files.
The RIAA is now attacking these networks using a strategy that could be called Crush the Connectors. A number of recent books on networks, such as Gladwell's The Tipping Point, Barabasi's Linked, and Watts' Six Degrees, have noted that large, loosely connected networks derive their effectiveness from a small number of highly connected nodes, a pattern called a Small World network. As a result, random attacks, even massive ones, typically leave the network only modestly damaged.
The flipside is that attacks that specifically target the most connected nodes are disproportionately effective. The RIAA's Crush the Connectors strategy will work, not simply because highly publicized legal action will deter some users, but because the value of the system will decay badly if the RIAA succeeds in removing even a small number of the best-provisioned nodes.
However, it will not work as well as the RIAA wants, even ignoring the public relations fallout, for two reasons. The first is that combining client, server, and router in one piece of software is not the last move available to network designers -- there is still the firewall. And the second is simply the math of popular music -- there are more people than songs.
Networks, Horizons, and Membranes
Napster was the last file-sharing system that was boundary-less by design. There was, at least in theory, one Napster universe at any given moment, and it was globally searchable. Gnutella, Kazaa, and other similar systems set out to decentralize even the address and search functions. This made these systems more robust in the face of legal challenges, but added an internal limit -- the search horizon.
Since such systems have no central database, they relay requests through the system from one node to the next. However, the "Ask two friends to ask two friends ad infinitum" search method can swamp the system. As a result, these systems usually limit the spread of search requests, creating an internal horizon. The tradeoff here is between the value of any given search (deeper searches are more effective) vs the load on the system as a whole (shallower searches reduce communications overhead.) In a world where the RIAA's attack mode was to go after central resources, this tradeoff worked well -- efficient enough, and resistant to Napster-style lawsuits.
However, these systems are themselves vulnerable in two ways -- first, anything that reduces the number of songs inside any given user's search horizon reduces the value of the system, causing some users to defect, which weakens the system still further. Second, because search horizons are only perceptual borders, the activity of the whole network can be observed by a determined attacker running multiple nodes as observation points. The RIAA is relying on both weaknesses in its current attack.
By working to remove those users who make a large number of files persistently available, the RIAA can limit the amount of accessible music and the trust the average user has in the system. Many of the early reports on the Crush the Connectors strategy suggest that users are not just angry with the RIAA, but with Kazaa as well, for failing to protect them.
The very fact that Crush the Connectors is an attack on trustworthiness, however, points to one obvious reaction: move from a system with search horizons to one with real membranes, and making those membranes social as well as technological.
Trust as a Border
There are several activities that are both illegal and popular, and these suffer from what economists call high transaction costs. Buying marijuana involves considerably more work than buying roses, in part because every transaction involves risk for both parties, and in part because neither party can rely on the courts for redress from unfair transactions. As a result, the market for marijuana today (or NYC tattoo artists in the 1980s, or gin in the 1920s, etc) involves trusted intermediaries who broker introductions.
These intermediaries act as a kind of social Visa system; in the same way a credit card issuer has a relationship with both buyer and seller, and an incentive to see that transactions go well, an introducer in an illegal transaction has an incentive to make sure that neither side defects from the transaction. And all parties, of course, have an incentive to avoid detection.
This is a different kind of border than a search horizon. Instead of being able to search for resources a certain topological distance from you, you search for resources a certain social distance from you. (This is also the guiding principle behind services like LinkedIn and Friendster, though in practice they represent their user's networks as being much larger than real-world social boundaries are.)
Such a system would add a firewall of sorts to the client, server, and router functions of existing systems, and that firewall would serve two separate but related needs. It would make the shared space inaccessible to new users without some sort of invitation from existing users, and it would likewise make all activity inside the space unobservable to the outside world.
Though the press is calling such systems "darknets" and intimating that they are the work of some sort of internet underground, those two requirements -- controlled membership and encrypted file transfer -- actually describe business needs better than consumer needs.
There are many ways to move to such membrane-bounded systems, of course, including retrofitting existing networks to allow sub-groups with controlled membership (possibly using email white-list or IM buddy-list tools); adopting any of the current peer-to-peer tools designed for secure collaboration (e.g. Groove, Shinkuro, WASTE etc); or even going to physical distribution. As Andrew Odlyzko has pointed out, sending disks through the mail can move enough bits in a 24 hour period to qualify as broadband, and there are now file-sharing networks whose members simply snail mail one another mountable drives of music.
A critical factor here is the social fabric -- as designers of secure networks know, protecting the perimeter of a network only works if the people inside the perimeter are trustworthy. New entrants can only be let into such a system if they are somehow vetted or vouched for, and the existing members must have something at stake in the behavior of the new arrivals.
The disadvantage of social sharing is simple -- limited membership means fewer files. The advantage is equally simple -- a socially bounded system is more effective than nothing, and safer than Kazaa.
If Kazaa, Gnutella and others are severely damaged by the Crush the Connectors attack, users will either give up free file-sharing, or switch to less efficient social spaces. This might seem like an unalloyed win for the RIAA, but for one inconvenient fact: there are more people than are songs.
There Are More People Than Songs
For the sake of round numbers, assume there are 500 million people using the internet today, and that much of the world's demand for popular music would be satisfied by the availability of something like 5 million individual songs (Apple's iTunes, by way of comparison, is a twentieth of that size.) Because people outnumber songs, if every user had one MP3 each, there would be a average of a hundred copies of every song somewhere online. A more realistic accounting would assume that at least 10% of the online population had at least 10 MP3 files each, numbers that are both underestimates, given the popularity of both ripping and sharing music.
Worse for the RIAA, the popularity of songs is wildly unequal. Some songs -- The Real Slim Shady, Come Away With Me -- exist on millions of hard drives around the world. As we've moved from more efficient systems like Napster to less efficient ones like Kazaa, it has become considerably harder to find bluegrass, folk, or madrigals, but not that much harder to find songs by Britney, 50 Cent, or John Mayer. And as with the shift from Napster to Kazaa, the shift from Kazaa to socially-bounded systems will have the least significant effect on the most popular music.
The worst news of all, though, is that songs are not randomly distributed. Instead, user clusters are a good predictor of shared taste. Make two lists, one of your favorite people and another of your favorite songs. What percentage of those songs could you copy from those people?
Both of those lists are probably in the dozens at most, and if music were randomly distributed, getting even a few of your favorite songs from your nearest and dearest would be a rare occurrence. As it is, though, you could probably get a significant percentage of your favorite songs from your favorite people. Systems that rely on small groups of users known to one another, trading files among themselves, will be less efficient than Kazaa or Napster, but far more efficient than a random distribution of music would suggest.
What Happens Next?
Small amounts of social file-sharing, by sending files as email attachments or uploading them to personal web servers, have always co-existed with the purpose-built file-sharing networks, but the two patterns may fuse as a result of the Crush the Connectors strategy. If that transition happens on a large scale, what might the future look like?
Most file-sharing would go on in groups from a half dozen to a few dozen -- small enough that every member can know every other member by reputation. Most file-sharing would take place in the sorts of encrypted workspaces designed for business but adapted for this sort of social activity. Some users would be members of more than one space, thus linking several cells of users. The system would be far less densely interconnected than Kazaa or Gnutella are today, but would be more tightly connected than a simple set of social cells operating in isolation.
It's not clear whether this would be good news or bad news for the RIAA. There are obviously several reasons to think it might be bad news: file-sharing would take place in spaces that would be much harder to inspect or penetrate; the lowered efficiency would also mean fewer high-yield targets for legal action; and the use of tools by groups that knew one another might make prosecution more difficult, because copyright law has often indemnified some types of non-commercial sharing among friends (e.g. the Audio Home Recording Act of 1992).
There is also good news that could come from such social sharing systems, however. Reduced efficiency might send many users into online stores, and users seeking the hot new song might be willing to buy them online rather than wait for the files to arrive through social diffusion, which would effectively turn at least some of these groups into buyers clubs.
The RIAA's reaction to such social sharing will be unpredictable. They have little incentive to seek solutions that don't try to make digital files behave like physical objects. They may therefore reason that they have little to lose by attacking social sharing systems with a vengeance. Whatever their reaction, however, it is clear that the current environment favors the development and adoption of social and collaborative tools, which will go on to have effects well outside the domain of file-sharing, because once a tool is adopted for one purpose, it often takes on a life of its own, as its users press such social tools to new uses.
posted by Arcadia at
6:57 AM
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home