Getting Started as an Entrepreneur – Wikibooks, open books for an open world.
via Getting Started as an Entrepreneur – Wikibooks, open books for an open world.
Author: escapebusinesssolutions
Unleashing the Ideavirus
Unleashing the Ideavirus 1 http://www.ideavirus.com
Unleashing the Ideavirus
By Seth Godin
Foreword by Malcolm Gladwell
©2000 by Do You Zoom, Inc.
You have permission to post this, email this, print this and pass it along for free to
anyone you like, as long as you make no changes or edits to its contents or digital
format. In fact, I’d love it if you’d make lots and lots of copies. The right to bind this
and sell it as a book, however, is strictly reserved. While we’re at it, I’d like to keep
the movie rights too. Unless you can get Paul Newman to play me.
Ideavirus™ is a trademark of Do You Zoom, Inc. So is ideavirus.com™.
Designed by Red Maxwell
You can find this entire manifesto, along with slides and notes and other good stuff, at
This version of the manifesto is current until September 17, 2000. After that date, please go
to http://www.ideavirus.com and get an updated version. You can buy this in book form on
September 1, 2000.
This book is dedicated to Alan Webber and Jerry Colonna. Of course.
Unleashing the Ideavirus 2 http://www.ideavirus.com
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 3 http://www.ideavirus.com
Look for the acknowledgments at the end. This is, after all, a new digital format, and you want to get right to it!
The #1 question people ask me after reading
Permission Marketing:
ÒSo, how do we get attention to ask for
permission in the first place?”
This manifesto is the answer to that question.
Unleashing the Ideavirus 4 http://www.ideavirus.com
Foreword
The notion that an idea can become contagious, in precisely the same way that a virus does,
is at once common-sensical and deeply counter-intuitive. It is common-sensical because all of
us have seen it happen: all of us have had a hit song lodged in our heads, or run out to buy a
book, or become infected with a particular idea without really knowing why. It is counterintuitive,
though, because it doesn’t fit with the marketer’s traditional vision of the world.
Advertisers spent the better part of the 20th century trying to control and measure and
manipulate the spread of information—to count the number of eyes and ears that they could
reach with a single message. But this notion says that the most successful ideas are those that
spread and grow because of the customer’s relationship to other customers—not the
marketer’s to the customer.
For years, this contradiction lay unresolved at the heart of American marketing. No longer.
Seth Godin has set out to apply our intuitive understanding of the contagious power of
information—of what he so aptly calls the ideavirus—to the art of successful
communication. “Unleashing the Ideavirus” is a book of powerful and practical advice for
businesses.
But more than that, it is a subversive book. It says that the marketer is not—and ought not
to be—at the center of successful marketing. The customer should be. Are you ready for that?
Malcolm Gladwell
Author
The Tipping Point
Unleashing the Ideavirus 5 http://www.ideavirus.com
Introduction
If you don’t have time to read the whole book, here’s what it says:
Marketing by interrupting people isn’t cost-effective anymore. You
can’t afford to seek out people and send them unwanted marketing
messages, in large groups, and hope that some will send you money.
Instead, the future belongs to marketers who establish a foundation
and process where interested people can market to each other. Ignite
consumer networks and then get out of the way and let them talk.
If you’re looking for mindblowing new ideas, you won’t find them in this, or any other
marketing book. Guerrilla marketing, 1:1 marketing, permission marketing—these ideas are
not really new, but they are thoughtful constructs that let you figure out how to do
marketing better. The fact is, if we built factories as badly as we create advertising campaigns,
the country would be in a shambles. This book will help you better understand the timehonored
marketing tradition of the ideavirus, and help you launch your own.
Questions the book answers:
1.
Why is it foolish to launch a new business with millions of dollars in TV ads?
2.
Are the market leaders in every industry more vulnerable to sudden successes by the
competition than ever before?
3.
Should book publishers issue the paperback edition of a book before the hardcover?
4.
What’s the single most important asset a company can create—and what is the simple
thing that can kill it?
5.
Every ad needs to do one of two things to succeed…yet most ads do neither. What’s the
right strategy?
6.
Does the Net create a dynamic that fundamentally changes the way everything is
marketed?
7.
How can every business…big and small…use ideavirus marketing to succeed?
Unleashing the Ideavirus 6 http://www.ideavirus.com
Foreword……………………………………………………………………………………………………………………….5
Introduction……………………………………………………………………………………………………………………6
SECTION 1: Why Ideas Matter…………………………………………………………………………………… 11
Farms, Factories And Idea Merchants ………………………………………………………………………………….12
Why Are Ideaviruses So Important?…………………………………………………………………………………….21
And Five Things Ideaviruses Have In Common……………………………………………………………………….22
Seven Ways An Ideavirus Can Help You: ……………………………………………………………………………… 23
The Sad Decline of Interruption Marketing ………………………………………………………………………….. 24
We Live In A Winner-Take-Almost-All World………………………………………………………………………….. 25
The Traffic Imperative: Why Sites Fail ……………………………………………………………………………….. 28
We Used To Make Food. We Used To Make Stuff. Now We Make Ideas…………………………………………. 30
People Are More Connected Than They Ever Were Before. We Have Dramatically More Friends Of Friends
And We Can Connect With Them Faster And More Frequently Than Ever……………………………………….31
ThereÕs A Tremendous Hunger To Understand The New And To Remain On The Cutting Edge……………34
While Early Adopters (The Nerds Who Always Want To Know About The Cool New Thing In Their Field)
Have Always Existed, Now WeÕve Got More Nerds Than Ever Before. If YouÕre Reading This, YouÕre A
Nerd!………………………………………………………………………………………………………………………….. 35
Ideas Are More Than Just Essays And Books. Everything From New Technology To New Ways Of Creating
To New Products Are Winning Because Of Intelligent Ideavirus Management By Their Creators………..36
The End Of The Zero Sum Game ………………………………………………………………………………………… 37
SECTION 2: How To Unleash An Ideavirus…………………………………………………………………..39
While It May Appear Accidental, ItÕs Possible To Dramatically Increase The Chances Your Ideavirus Will
Catch On And Spread. ……………………………………………………………………………………………………..40
The Heart Of The Ideavirus: Sneezers ………………………………………………………………………………….41
Sneezers Are So Important, We Need To Subdivide Them………………………………………………………… 42
The Art Of The Promiscuous …………………………………………………………………………………………….. 47
ItÕs More Than Just Word Of Mouth ……………………………………………………………………………………..51
An Ideavirus Adores A Vacuum …………………………………………………………………………………………. 52
Unleashing the Ideavirus 7 http://www.ideavirus.com
Once It Does Spread, An Ideavirus Follows A Lifecycle. Ignore The Lifecycle And The Ideavirus Dies Out.
Feed It Properly And You Can Ride It For A Long Time…………………………………………………………….54
Viral Marketing Is An Ideavirus, But Not All Ideaviruses Are Viral Marketing ……………………………….. 55
What Does It Take To Build And Spread An Ideavirus? ……………………………………………………………. 57
There Are Three Key Levers That Determine How Your Ideavirus Will Spread:………………………………60
Ten Questions Ideavirus Marketers Want Answered ………………………………………………………………. 64
Five Ways To Unleash An Ideavirus ……………………………………………………………………………………. 65
SECTION THREE: The Ideavirus Formula ……………………………………………………………………. 78
Managing Digitally-Augmented Word Of Mouth……………………………………………………………………… 79
Tweak The Formula And Make It Work …………………………………………………………………………………80
Advanced Riffs On The Eight Variables You Can Tweak In Building Your Virus………………………………. 85
Hive …………………………………………………………………………………………………………………………… 88
Velocity………………………………………………………………………………………………………………………. 92
Vector………………………………………………………………………………………………………………………… 94
Medium ………………………………………………………………………………………………………………………. 96
SMOOTHNESS: It Would All Be Easy If We Had Gorgons……………………………………………………………. 98
Persistence …………………………………………………………………………………………………………………100
Amplifier …………………………………………………………………………………………………………………….102
SECTION 4: Case Studies and Riffs…………………………………………………………………………..104
The Vindigo Case Study…………………………………………………………………………………………………..105
Saving The World With An Ideavirus ………………………………………………………………………………….. 107
Moving Private To Public…………………………………………………………………………………………………..111
YouÕre In The Fashion Business! ………………………………………………………………………………………..113
The Money Paradox ………………………………………………………………………………………………………..117
Think Like A Music Executive (Sometimes)…………………………………………………………………………..119
Is That Your Final Answer?……………………………………………………………………………………………….121
A Dozen ideaviruses Worth Thinking About…………………………………………………………………………. 123
Why I Love Bestseller Lists………………………………………………………………………………………………124
How A Parody Of Star Wars Outsold Star Wars …………………………………………………………………….. 127
Unleashing the Ideavirus 8 http://www.ideavirus.com
Wassup? ……………………………………………………………………………………………………………………..129
Judging a book by its cover ……………………………………………………………………………………………..131
Being The Most ……………………………………………………………………………………………………………. 133
In Defense Of World Domination ………………………………………………………………………………………. 135
If YouÕre A Member Of The Academy, You Go To Movies For Free …………………………………………….. 137
How An Ideavirus Can Drive The Stock Market …………………………………………………………………….. 139
Bumper Sticker Marketing……………………………………………………………………………………………….142
No, You Go First! ………………………………………………………………………………………………………….. 143
Digital Media Wants to Be Free…………………………………………………………………………………………145
Van Gogh Lost His Ear To Prove A Point ……………………………………………………………………………..148
Answering InaÕs Question………………………………………………………………………………………………..150
Crossing The Chasm With An Ideavirus ……………………………………………………………………………….152
The Myth Of The Tipping Point ………………………………………………………………………………………….156
The Compounding Effect …………………………………………………………………………………………………158
Bill GatesÕ Biggest Nightmare…………………………………………………………………………………………..160
Hey, Skinny!…………………………………………………………………………………………………………………164
Get Big Fast? The Mistake So Many Companies MakeÉ…………………………………………………………..165
The Heart Of Viral Marketing……………………………………………………………………………………………168
The Great Advertising Paradox………………………………………………………………………………………….171
Permission: The Missing Ingredient…………………………………………………………………………………… 174
How A Virus And Permission Team Up To Find Aliens…………………………………………………………….. 176
The Art of Creating an Ideavirus………………………………………………………………………………………. 177
Is He Really More Evil Than Satan Himself? ………………………………………………………………………… 178
Case Study: Why Digimarc Is Going To Fail…………………………………………………………………………..179
Why Are These Cows Laughing?…………………………………………………………………………………………181
Never Drink Alone …………………………………………………………………………………………………………183
The Power Of Parody ……………………………………………………………………………………………………..185
Bee Stings And The Measles …………………………………………………………………………………………….186
But IsnÕt It Obvious?………………………………………………………………………………………………………187
Unleashing the Ideavirus 9 http://www.ideavirus.com
Your CompanyÕs Worst Enemy ………………………………………………………………………………………….189
Step By Step, Ideavirus Tactics: ……………………………………………………………………………………….192
The Future Of The Ideavirus: What Happens When Everyone Does It? ………………………………………..194
Acknowledgments ………………………………………………………………………………………………..196
Unleashing the Ideavirus 10 http://www.ideavirus.com
SECTION 1: Why Ideas Matter
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 11 http://www.ideavirus.com
Farms, Factories And Idea Merchants
Imagine for a second that you’re at your business school reunion, trading lies and bragging
about how successful you are and are about to become. Frank the jock talks about the dotcom
company he just started. Suzie the ex-banker is now focusing her energy on rebuilding
Eastern Europe. And then the group looks at you. With a wry look of amusement, you
answer:
“Well, the future—the really big money—is in owning a farm. A small one, maybe 100
acres. I intend to invest in a tractor of course, and expect that in just a few years my husband
and I can cash out and buy ourselves a nice little brownstone in the city.”
Ludicrous, no? While owning a farm may bring tremendous lifestyle benefits, it hasn’t been a
ticket to wealth for, say, 200 years.
What about owning a factory then? Perhaps the road to riches in the new economy would be
to buy yourself a hot-stamping press and start turning out steel widgets. Get the UAW to
organize your small, dedicated staff of craftsmen and you’re on your way to robber-baron
status.
Most of us can agree that the big money went out of owning a factory about thirty years ago.
When you’ve got high fixed costs and you’re competing against other folks who also know
how to produce both quantity and quality, unseemly profits fly right out the window.
Fact is, the first 100 years of our country’s history were about who could build the biggest,
most efficient farm. And the second century focused on the race to build factories. Welcome
to the third century, folks. The third century is about ideas.
Alas, nobody has a clue how to build a farm for ideas, or even a factory for ideas. We
recognize that ideas are driving the economy, ideas are making people rich and most
important, ideas are changing the world. Even though we’re clueless about how to best
organize the production of ideas, one thing is clear: if you can get people to accept and
Unleashing the Ideavirus 12 http://www.ideavirus.com
embrace and adore and cherish your ideas, you win. You win financially, you gain power and
you change the world in which we live.
So how do you win? What do you need to do to change the status quo of whatever industry
you’re in, or, if you’re lucky, to change the world?
If you’re a farmer, you want nothing more than a high price for your soybeans. If you’re a
manufacturer of consumer goods, you want a display at the cash register at Wal-Mart. But
what if you’re an idea merchant?
The holy grail for anyone who trafficks in ideas is this: to unleash an ideavirus.
An idea that just sits there is worthless. But an idea that moves and grows and infects
everyone it touches… that’s an ideavirus.
In the old days, there was a limit on how many people you could feed with the corn from
your farm or the widgets from your factory. But ideas not only replicate easily and well, they
get more powerful and more valuable as you deliver them to more people.
How does an ideavirus manifest itself? Where does it live? What does it look like? It’s useful
to think of ideas of every sort as being similar. I call them manifestos. An idea manifesto is a
powerful, logical “essay” that assembles a bunch of existing ideas and creates a new one.
Sometimes a manifesto is a written essay. But it can be an image, a song, a cool product or
process… the medium doesn’t matter. The message does. By lumping all sorts of
ideas—regardless of format—into the same category (manifestos) it’s much easier to think of
them as versions of the same thing. As long as you can use your manifesto to change the way
people think, talk or act… you can create value.
Definition: MEDIUM In order to move, an idea has to be encapsulated in a medium. It
could be a picture, a phrase, a written article, a movie, even a mathematical formula (e=mc2).
The Medium used for transmitting the ideavirus determines how smooth it is as well as the
velocity of its growth. A medium is not a manifesto—every idea is a manifesto, trying to
make its point, and the medium is the substance that the idea lives in.
Unleashing the Ideavirus 13 http://www.ideavirus.com
Not only is this an essay about ideas and ideaviruses…it’s also a manifesto striving to become
an ideavirus! If this manifesto changes your mind about marketing and ideas, maybe you’ll
share it with a friend. Or two. Or with your entire company. If that happens, this idea will
become an ideavirus, and spread and gain in value.
We live in a world where consumers actively resist marketing. So it’s imperative to stop
marketing at people. The idea is to create an environment where consumers will market to
each other.
Is an ideavirus a form of marketing? Sure it is. And today, marketing is all there is. You don’t
win with better shipping or manufacturing or accounts payable. You win with better
marketing, because marketing is about spreading ideas, and ideas are all you’ve got left to
compete with.
The future belongs to the people who unleash ideaviruses.
What’s an ideavirus? It’s a big idea that runs amok across the target audience. It’s a
fashionable idea that propagates through a section of the population, teaching and changing
and influencing everyone it touches. And in our rapidly/instantly changing world, the art
and science of building, launching and profiting from ideaviruses is the next frontier.
Have you ever heard of Hotmail? Ever used it? If so, it’s not because Hotmail ran a lot of TV
ads (they didn’t). It’s because the manifesto of free email got to you. It turned into an
ideavirus. Someone you know and trust infected you with it. What about a Polaroid
camera… was your first exposure (no pun intended!) in a TV ad, or did you discover it when
a friend showed you how cool the idea of an instant photograph was?
Sometimes it seems like everyone is watching the same TV show as you, or reading the same
book, or talking about the same movie or website. How does that happen? It usually occurs
because the idea spreads on its own, through an accidental ideavirus, not because the
company behind the product spent a ton of money advertising it or a lot of time
Unleashing the Ideavirus 14 http://www.ideavirus.com
orchestrating a virus. And how the idea spreads, and how to make it spread faster—that’s the
idea behind unleashing an ideavirus.
Word of mouth is not new—it’s just different now. There were always ideaviruses—gossip or
ideas or politics that spread like wildfire from person to person. Without running an ad or
buying a billboard, Galileo managed to upset all of Pisa with his ideas. Today, though,
ideaviruses are more important and more powerful than ever. Ideaviruses are easier to launch
and more effective. Ideaviruses are critical because they’re fast, and speed wins and speed
kills—brands and products just don’t have the time to develop the old way. Ideaviruses give
us increasing returns—word of mouth dies out, but ideaviruses get bigger. And finally,
ideaviruses are the currency of the future. While ideaviruses aren’t new, they’re important
because we’re obsessed with the new, and an ideavirus is always about the new.
Remember the slogan, “Only her hairdresser knows for sure?” That was classic brand
marketing, and it flew in the face of word of mouth. It was an ad for a product that was
supposed to be a secret—a secret between you, your hairdresser and Clairol.
A few years later, Herbal Essence took a totally different tack… they tried to encourage you
to tell your friends. But while word of mouth works great among the people who use a
product and their immediate friends—if I love your story or hate your service, I’ll tell a few
friends—it dies out fast. There’s no chance a friend of a friend is going to tell you about my
horrible experience on United Airlines or how much I loved flying on Southwest. Word of
mouth fades out after a few exchanges.
But now, aided by the Net and abetted by the incredible clutter in our universe, ideaviruses
are spreading like wildfire. We’re all obsessed with ideas because ideas, not products, are the
engine of our new economy.
I wore Converse sneakers growing up… so did you. But the shareholders of Converse never
profited from the idea of the shoe—they profited from the manufacture of a decent sneaker.
If two sneakers were for sale, you bought the cheaper one.
Unleashing the Ideavirus 15 http://www.ideavirus.com
It took Converse generations to build a brand and years to amortize a factory and they were
quite happy to extract a modest profit from every pair of sneakers sold, because Converse
knew their factory would be around tomorrow and the day after that. So sneakers, like
everything else, were priced by how much they cost, and sold one pair at a time by earnest
shoe salesmen who cared about things like how well the shoes fit.
Converse could take their time. They were in this for the long haul. Those days are long
gone. Twenty years later, it’s the idea of Air Jordan sneakers, not the shoe, that permits Nike
to sell them for more than $100. It’s the sizzle, not the fit. The idea makes Nike outsized
profits. And Nike knows that idea won’t last long, so they better hurry—they need another
ideavirus, fast.
In the old days, we used to sneer at this and call it a fad. Today, everything from presidential
politics to music to dentistry is driven by fads—and success belongs to marketers who
embrace this fact.
Source: Forrester Research
It took 40 years for radio to have ten million users. By then, an industry had grown that
could profit from the mass audience. It took 15 years for TV to have ten million users. It
Unleashing the Ideavirus 16 http://www.ideavirus.com
only took 3 years for Netscape to get to 10 million, and it took Hotmail and Napster less
than a year. By aggregating mass audiences to themselves (and not having to share them with
an entire industry), companies like Netscape and Hotmail are able to realize huge profits,
seemingly overnight. And they do it by spreading ideaviruses.
Ideas can now be carried in the ether. Because the medium for carrying ideas is fast and
cheap, ideas move faster and cheaper! Whether it’s the image of the new VW Beetle (how
long did it take for the idea of that car to find a place in your brain?) or the words of a new
Stephen King novel (more than 600,000 people read it in the first week it was available
online), the time it takes for an idea to circulate is approaching zero.
Why should we care? Why does it matter that ideas can instantly cross international
boundaries, change discussions about politics, crime and justice or even get us to buy
something? Because the currency of our future is ideas, and the ideavirus mechanism is the
way those ideas propagate. And the science and art of creating ideaviruses and using them for
profit is new and powerful. You don’t have to wait for an ideavirus to happen organically or
accidentally. You can plan for it and optimize for it and make it happen.
Sure, some ideaviruses are organic. They happen and spread through no overt action or
intent on the part of the person who creates them (the Macarena wasn’t an organized plot…
it just happened). Others, though, are the intentional acts of smart entrepreneurs and
politicians who know that launching and nurturing an ideavirus can help them accomplish
their goals.
In the old days, the way we sold a product was through interruption marketing. We’d run
ads, interrupt people with unanticipated, impersonal, irrelevant ads and hope that they’d buy
something. And sometimes, it worked.
The advantage of this branding strategy is that the marketer is in complete and total control.
The disadvantage is that it’s hard and expensive. Every time a catalog clothier (Land’s End,
Eddie Bauer, you name it) wants to sign up a new customer, they need to buy a few hundred
stamps, send out some carefully designed catalogs and hope that one person sends them
money.
Unleashing the Ideavirus 17 http://www.ideavirus.com
What marketers are searching for is a way to circumvent the tyranny of cost-per-thousand
interruptions. They need something that ignites, a way to tap into the invisible currents that
run between and among consumers, and they need to help those currents move in better,
faster, more profitable ways. Instead of always talking to consumers, they have to help
consumers talk to each other.
A beautifully executed commercial on the Super Bowl is an extraordinarily risky bet.
Building a flashy and snazzy website is almost certain to lead to failure. Hiring a celebrity
spokesperson might work on occasion, but more often than not, it won’t break through the
clutter. Whenever advertisers build their business around the strategy of talking directly to
the customer, they become slaves to the math of interruption marketing.
In traditional interruption marketing, the marketer talks directly to as many consumers as possible, with no
intermediary other than the media company. The goal of the consumer is to avoid hearing from the advertiser. The
goal of the marketer is to spend money buying ads that interrupt people who don’t want to be talked to!
Unleashing the Ideavirus 18 http://www.ideavirus.com
In creating an ideavirus, the advertiser creates an environment in which the idea can replicate and spread. It’s the
virus that does the work, not the marketer.
Fortunately, there are already proven techniques you can use to identify, launch and profit
from ideas that can be turned into viruses. There’s a right and a wrong way to create them,
and more important, the care and feeding of your ideavirus can dramatically affect its
potency.
One of the key elements in launching an ideavirus is concentrating the message. If just 1% or
even 15% of a group is excited about your idea, it’s not enough. You only win when you
totally dominate and amaze the group you’ve targeted. That’s why focusing obsessively on a
geographic or demographic or psychographic group is a common trait among successful idea
merchants.
Why are new companies launching on the Net so obsessed with traffic and visitors? Why is a
company like GeoCities sold for more than $2 billion, when it has close to zero revenue and
interesting, but by no means unique, software?
Because infecting large populations with the ideavirus is the first step to building a profitable
business model. The key steps for Internet companies looking to build a virus are:
Unleashing the Ideavirus 19 http://www.ideavirus.com
1.
Create a noteworthy online experience that’s either totally new or makes the user’s life
much better. Or make an offline experience better/faster/cheaper so that switching is
worth the hassle.
2.
Have the idea behind your online experience go viral, bringing you a large chunk of the
group you’re targeting WITHOUT having to spend a fortune advertising the new
service.
3.
Fill the vacuum in the marketplace with YOUR version of the idea, so that competitors
now have a very difficult time of unteaching your virus and starting their own.
4.
Achieve “lock in” by creating larger and larger costs to switching from your service to
someone else’s.
5.
Get permission from users to maintain an ongoing dialogue so you can turn the original
attention into a beneficial experience for users and an ongoing profit stream for you.
6.
Continue creating noteworthy online experiences to further spread new viruses, starting
with your core audience of raving fans.
Unleashing the Ideavirus 20 http://www.ideavirus.com
Why Are Ideaviruses So Important?
1.
We live in a winner-take-almost-all world. (Zipf’s law.)
2.
We used to focus on making food. We used to make stuff. Now we make ideas.
3.
People are more connected than ever. Not only are we more aware that our friends have
friends but we can connect with them faster and more frequently.
4.
There’s a tremendous hunger to understand the new and to remain on the cutting edge.
5.
While early adopters (the nerds who always want to know about the cool new thing in
their field) have always existed, now we’ve got more nerds than ever. If you’re reading this,
you’re a nerd!
6.
The profit from creating and owning an ideavirus is huge.
Unleashing the Ideavirus 21 http://www.ideavirus.com
And Five Things Ideaviruses Have In Common
1.
The most successful ideaviruses sometimes appear to be accidents, but it is possible to
dramatically increase the chances your ideavirus will catch on and spread.
2.
An ideavirus adores a vacuum. (This is a big idea. Read on to see what I mean).
3.
Once an ideavirus spreads, it follows a lifecycle. Ignore the lifecycle and the ideavirus dies
out. Feed it properly and you can extend its useful life and profit from it for a long time.
4.
Ideaviruses are more than just essays and books. Everything from new technology to new
ways of creating new products are winning because of intelligent seeding by their
creators.
5.
Viral marketing is a special case of an ideavirus. Viral marketing is an ideavirus in which
the carrier of the virus IS the product.
Unleashing the Ideavirus 22 http://www.ideavirus.com
Seven Ways An Ideavirus Can Help You:
1.
When everyone in town tells ten friends about your amazing ice cream
shop and a line forms out the door (supercharged word of mouth due to
the virus having dominated the town so completely).
2.
When your company’s new mass storage format catches on and it
becomes the next Zip drive.
3.
When an influential sports writer names your daughter as a high school
All-American basketball player and coaches line up outside the door
with scholarships.
4.
When Steve Jobs commissions the iMac, which spreads the word about
the Mac faster than any advertising ever could, raising market share and
saving your favorite computer company from bankruptcy.
5.
When you write a report for your boss about how your company should
deal with an opportunity in Cuba and it gets passed on, from person to
person, throughout the company, making you a hero and a genius.
6.
When the demo recording you made becomes a bestseller on MP3.com
and you get a call from Sony, who wants to give you a recording
contract.
7.
When you are able to devise a brand-new Internet business plan for a
product that’s useful and also embodies viral marketing…growing from
nothing to a million users in a month and making you rich along the
way.
Unleashing the Ideavirus 23 http://www.ideavirus.com
The Sad Decline of Interruption Marketing
When I first starting writing about Permission Marketing about four years ago, much of
what I said was considered heresy. “What do you mean TV ads are going to decline in
effectiveness?” “How dare you say anything negative about banner ads—of course they
work!” or “Direct mail has never been healthier!”
History, fortunately for me, has borne out my cries of doom and gloom about interruption
marketing. The TV networks are diversifying away from their traditional network TV
business as fast as they can. Banner clickthrough rates are down 85% or more. Ads are
sprouting up on the floors of the supermarket, in the elevator of the Hilton hotel in Chicago
and even in urinals. And everywhere you look, unanticipated, impersonal and irrelevant ads
are getting more expensive and less effective.
There’s a crisis in interruption marketing and it’s going to get much worse. It took more
than thirty pages to build the case against this wasteful, costly ($220 billion a year)
outmoded expense in Permission Marketing, so I’ll only spend a page on it here. If you want
to read the entire jeremiad, send a note to free@permission.com and I’ll send it to you for
free.
Unless you find a more cost-effective way to get your message out, your business is doomed.
You can no longer survive by interrupting strangers with a message they don’t want to hear,
about a product they’ve never heard of, using methods that annoy them. Consumers have
too little time and too much power to stand for this any longer.
Unleashing the Ideavirus 24 http://www.ideavirus.com
We Live In A Winner-Take-Almost-All World
Quick! Name an oil painting hanging in a museum somewhere in the world.
Did you say, “the Mona Lisa”?
As I walk through the Louvre, arguably one of the top ten most packed-with-high-qualitypaintings
museums on the planet, I pass one empty room after another, then come to an
alcove packed with people. Why? Why are these people clawing all over each other in order
to see a painting poorly displayed behind many inches of bullet-proof glass?
The reason the Mona Lisa is the most famous painting in the world is
that something had to be the most famous painting in the world and it
might as well be the Mona Lisa.
Busy people don’t have time to look at every painting. They only have
room in their overcrowded, media-hyped brains for a few paintings.
And when you come right down to it, most people would like to see only the “celebrity”
paintings. And just as there can only be one “My most favorite famous actress” (Julia
Roberts) and one “this site equals the Internet” (Yahoo!), there’s only room for one “most
famous painting in the world” and the safe choice is the Mona Lisa.
There’s a name for this effect. It’s called Zipf’s law, after George Kingsley Zipf (1902-1950),
a philologist and professor at Harvard University. He discovered that the most popular word
in the English language (“the”) is used ten times more than the tenth most popular word,
100 times more than the 100th most popular word and 1,000 times more than the 1,000th
most popular word.
Unleashing the Ideavirus 25 http://www.ideavirus.com
It’s also been discovered that this same effect applies to market share for software, soft drinks,
automobiles, candy bars, and the frequency of hits on pages found on a website. The chart
above shows actual visits to the different pages at Sun’s website.
In almost every field of endeavor, it’s clear that being #1 is a lot better than being #3 or #10.
There isn’t an even distribution of rewards, especially in our networked world.
On the Net, the stakes are even larger. The market capitalization of Priceline, eBay and
Amazon approaches 95% of the total market capitalization of every other consumer ecommerce
stock combined. Clearly, there’s a lot to be gained by winning.
An ideavirus lets you make something like this happen to your idea, to your business, to your
product. While the benefits of being #1 for a public Internet stock or an oil painting are
clear, it’s just as important to small businesses and individuals.
Ideaviruses are faced with a brickwall filter. In electronics, a brickwall filter wipes out certain
frequencies and lets the rest through. There’s no room for second place or extra
effort—either you’re in or you’re out. Ideaviruses are win/lose propositions. Either the
velocity and smoothness are high enough that it becomes a bonafide epidemic, or they’re not
and it dies out. Either your ideavirus works or it doesn’t. Smart propagators know when to
quit if their ideavirus isn’t getting through the filter.
Definition: VELOCITY
The velocity is a measure of how fast the idea spreads from
one party to another. If an idea is going to hit ten people before it gets to me, the multiplier
effect is large indeed—fast steps lead to more people being infected before it dies out.
Unleashing the Ideavirus 26 http://www.ideavirus.com
Definition: SMOOTHNESS SMOOTHNESS
How easy is it for an end user to spread this particular
ideavirus? Can I click one button or mention some magic phrase, or do I have to go through
hoops and risk embarrassment to tell someone about it?
For example, it’s pretty easy to talk about your hairdresser. Someone tells you you’ve got a
great haircut, and you say, “Yeah, I went to Bob at Bumble & Bumble.” On the other hand,
spreading the word about your reflexology therapist is pretty tricky. You’re not sure when to
bring it up, and you really don’t have words to describe it.
The smoothest viruses, like Hotmail, spread themselves. Just the act of using the product
spreads the virus. There’s an obvious relationship between smoothness and catchiness. A
product that’s easy to recommend is often a product that’s easy to get hooked on.
Eric Raymond was a little known programmer when he wrote an essay called “The Cathedral
and the Bazaar.” It was a manifesto—an essay designed to become an ideavirus—arguing
why the open source approach to coding (creating stuff like Linux) made sense. But instead
of having a magazine or a book publisher bring it to market, he posted the essay online, in
text, postscript and audio form. And he gave it away for free.
Within months, tens of thousands of people had read it. Months after that, Raymond
published this essay with some of his other free essays in a book. That book became an
“instant” bestseller. Of course, it wasn’t instant at all. He had laid the foundation long
before, by building an ideavirus.
So, what has creating an ideavirus done for Raymond’s value? Let’s take a crass look at his
financial situation: The virus led to increased demand for his services as a programmer (he
can pick his jobs if he likes), as a consultant, and even as a public speaker. The last I saw, he
had just written an essay about what it was like to make a fortune during an IPO!
Unleashing the Ideavirus 27 http://www.ideavirus.com
The Traffic Imperative: Why Sites Fail
A site without traffic doesn’t exist.
According to Forrester Research, only 20% of 50 leading online retailers expect to turn a
profit this year. Just 18% more expect to be profitable next year. It’s becoming increasingly
obvious that many of these sites will never turn a profit, and that they’re hoping to last long
enough to be acquired or sell their stock.
A recent McKinsey and Co. study found that the vast majority of online retailers are not
only unprofitable, they’re actually losing money on every sale. Without even computing the
cost of advertising and clicks, these sites have discounted their prices so significantly that the
contribution margin from each sale is negative. The average online drugstore, for example,
loses $16.42 on each and every sale, before computing the cost of traffic.
Why? Many of these sites are confusing low prices with an effective customer acquisition
tool. There’s probably no way that’s less effective and more costly than cutting your prices to
the point where you lose money on each sale (for Amazon naysayers—they actually make a
profit of about $5 on the average book order).
Add to this mess the obscene cost of customer acquisition—estimated by the Boston
Consulting Group to be more than $80 a visitor (that’s for visitors, not even customers) for
most online merchants. Now you can see the huge hurdle these sites are going to have to
cross in order to be profitable.
This problem isn’t unique to the online world, of course. When I was enrolled at Tufts
University in 1980, there were two homemade ice cream stores within two miles of campus.
One was Joey’s, which made a terrific product (they used Hydrox cookies instead of Oreos,
by the way, so you could avoid the animal fat if you wanted) and there was never, ever a line.
Unleashing the Ideavirus 28 http://www.ideavirus.com
In the other direction was the now famous Steve’s Ice Cream. His prices were a bit higher
than Joey’s, but his profits were clearly much higher. Why? Because there was always a line at
Steve’s. A long line. Sometimes you’d wait an hour to get an ice cream cone.
What happened? Why did one ice cream shop go viral and the other languished at the edge
of profitability? It certainly wasn’t about advertising, because neither shop did any. The
reason Steve Herrell’s shop did so well is that it was famous for having a line! People brought
folks from out of town to have the experience. Locals came back because they’d convinced
themselves that if the hive liked it enough to wait an hour for an ice cream cone, well, it
must be worth it. Suddenly, it wasn’t about the ice cream. It was about the experience.
Most online merchants, being risk averse copycats afraid to innovate, are guaranteeing that
there will be no ideavirus created around their businesses. By paying millions to AOL and
Yahoo! for “traffic,” they’re investing in exactly the wrong sort of buzz. The
alternative—focusing on people who can promote your site, affiliate programs, unique
promotions and building wow, zing and magic into the site—is just too much work for most
sites.
Unleashing the Ideavirus 29 http://www.ideavirus.com
We Used To Make Food. We Used To Make Stuff. Now We Make Ideas.
Here are some astonishing facts you should think about long and hard on your way to work
tomorrow:
Twenty years ago, the top 100 companies in the Fortune 500 either dug something out of
the ground or turned a natural resource (iron ore or oil) into something you could hold.
Today, fewer than half of the companies on the list do that. The rest make unseemly profits
by trafficking in ideas.
In 1998, there were 30,000 new musical CDs published, including one from the Pope (his,
which I like a lot, features a little rap, a little techno and a lot of worldbeats).
Ninety-nine percent of Yahoo’s market capitalization is due to brand, sizzle, user loyalty and
other “soft” ideas. Only 1% of the company’s value is due to actual unique stuff that you
can’t get anywhere else.
Nathan Mhyrvold, former chief scientist at Microsoft, says a great programmer is worth
10,000 times more than an average one. Why? Because of the quality of her ideas.
The important takeaway is this: Ideas aren’t a sideshow that make our factory a little more
valuable. Our factory is a sideshow that makes our ideas a little more valuable!
Unleashing the Ideavirus 30 http://www.ideavirus.com
People Are More Connected Than They Ever Were Before. We Have
Dramatically More Friends Of Friends And We Can Connect With Them Faster
And More Frequently Than Ever.
Think back. Really far. Ten years ago.
How many people did you have regular telephone contact with ten years ago? Probably ten
or twenty or thirty in your personal life, and maybe 100 at work?
Now, take a look at your email inbox and your ICQ (the most popular instant messenger
program) buddy list. How many people do you hear from every week?
We’re far more connected than we ever were. And now, we’ve got second or third or fourth
order connections. There’s an email in my box from someone who is married to someone I
went to summer camp with twenty years ago who got my email address from a third friend.
Another message is from a former employee, telling me about a doctor who’s about to lose
his license for trying radical medical treatments, and how her mother-in-law will suffer if this
guy can’t practice any longer.
It’s hard for me to imagine either person contacting me if they had to walk across the village
and bang on the door of my hut or pick up the phone and call me. But the moment you
connect to the Internet, you connect, at some level, to all of us. And the connections make
ideas travel. Fast.
What’s the difference between word of mouth and an ideavirus? Two differences. First, word
of mouth tends to spread slower, be more analog. If you like a book, you might tell a friend
or two. And then your friends are unlikely to tell someone else until they read it for
themselves.
Second, word of mouth dies off. Because the numbers are smaller, it doesn’t take many
people who don’t participate in the word of mouth for each generation to be smaller than the
one before it.
Unleashing the Ideavirus 31 http://www.ideavirus.com
Here’s a schematic of typical word of mouth. Notice how few cycles there are, and how it drops off
over time.
Here’s an ideal ideavirus. Note how much more frequently the cycles occur, and how each cycle sees
the virus grow.
With an ideavirus, both principles no longer apply. Ideaviruses spread fast and they spread
far. With word of mouse (word of mouth augmented by the power of online
communication), you can tell 100 friends, or a thousand friends. Because the numbers are
larger and faster, the virus grows instead of slows.
Even before the Net, there were special cases of viruses. In traditional word of mouth in the
book business, someone reads a book and tells a friend. It’s nice, but it’s not usually enough.
The Bridges of Madison County, however, became the bestselling novel of the decade, because
booksellers adopted it and told people. As a bookseller, you’ve got exposure not just to a few
people, but hundreds of people. So the serendipitous word of mouth that helps some books
is replaced by a rapid, virulent alternative.
Unleashing the Ideavirus 32 http://www.ideavirus.com
On the other hand, most Americans have never had a massage from a professional masseuse.
Why? Because in order to understand the power of a massage, you have to get one. We don’t
currently have the word or picture tools to adequately describe the positive benefits of a
massage, and just as important, there isn’t a powerful spokesperson for massage who has
spent the time and energy to develop the ideavirus. There’s no real medium to transmit the
message. So the message travels slowly. So there is no virus around the idea of a massage.
Unleashing the Ideavirus 33 http://www.ideavirus.com
ThereÕs A Tremendous Hunger To Understand The New And To Remain On The
Cutting Edge.
Jed Clampett discovered that finding oil on his property was a sure road to riches. Today, the
road seems to be paved with awareness. If you know what’s news, if you know what’s the
latest, hottest, most impactful new idea, it’s much easier to succeed. You can profit in the
stock market, do better in politics, find breakthroughs in science, or programming or
marketing.
Why does this matter? Because in a society where the new isn’t valued, your social standing
doesn’t increase when you become a nerd. And because ideaviruses are really nothing but
amplified gossip about new stuff, they can’t take root in a culture that doesn’t care about the
new.
Take a look at the Top 40 charts in Billboard magazine. Thirty or forty years ago, a record
could easily stay on the list for six months or more. Today, new records come and go much
faster. Why? Because we are happily saturated in the current hit, and then move on.
Last year, 1,778 business books were published in the U.S. alone. Every one of them got read
by someone, some by an awful lot of people. Why? Because as our world changes faster and
faster and faster, knowing is just as important as having. And that makes the population ready
and eager for the next ideavirus.
As the speed of new ideas entering the community has increased, so has our respect for
people who know. And because it’s valuable, we’re open to both hearing about the new and
telling others about it.
Unleashing the Ideavirus 34 http://www.ideavirus.com
While Early Adopters (The Nerds Who Always Want To Know About The Cool
New Thing In Their Field) Have Always Existed, Now WeÕve Got More Nerds
Than Ever Before. If YouÕre Reading This, YouÕre A Nerd!
The Internet turned us all into nerds. AltaVista isn’t cool any more—google.com is. Don’t
use the Palm, that’s passé. Try this Handspring instead. Suddenly we’re ready, willing and
able to be at the bleeding edge, all the time.
The profit from creating and owning an ideavirus is huge, huger than it ever was before. It
used to be that only a few stereotypical nerds cared about the latest pocket calculator. Today,
you’ll see people talking about their handheld computer on the subway. It used to be that
only a few people knew about the latest Salsa hit out of Mexico or the coolest new chef in
Los Angeles. Today, the roles are totally reversed. Your parents are nerds!
It’s not just that our society is rewarding people who are sensitive enough or smart enough or
cool enough to know about the next new thing. It’s that many of us have crossed over a line
and gone from being the vast majority who waited for something to become
mainstream—we’ve become the early adopters, the folks on the bleeding edge who actually
seek out innovation. The combined circulation of Wired, Fast Company and PC Magazine is
rapidly approaching the total circulation of Sports Illustrated.
Because the population has shifted, the sweet spot has shifted. Companies no longer make
most of their money harvesting money from the laggards who finally get around to buying
something at K-Mart. They make their money the first day, the first week, the first month an
idea is out there.
If something is new and different and exciting and getting buzzed about, we want to know
about it, be part of it. The fashion is now to be in fashion, and ideas are the way we keep up.
Unleashing the Ideavirus 35 http://www.ideavirus.com
Ideas Are More Than Just Essays And Books. Everything From New
Technology To New Ways Of Creating To New Products Are Winning Because
Of Intelligent Ideavirus Management By Their Creators.
A manifesto is a carefully organized series of ideas, designed to get someone to come around
to your point of view. But while one way to make a complicated argument is with a book,
you can just as easily (and sometimes more effectively) send it through a song (Bob Dylan
did this for Hurricane Carter) or with something as elegant as an OXO vegetable peeler.
When you first see the OXO, you instantly understand the idea behind it. You just know it
will work better and cut you less often. If you’ve ever peeled a vegetable, you want an OXO.
The design of the OXO is quite simply a manifesto that says, “There’s a smart, comfortable
way to do this annoying task.” Is the OXO going to get viral? Not across the general
population, of course, but if you hang out with a group of people who have arthritis or love
kitchen stuff, it already has. Just take a look at the glowing reviews of this peeler on
Amazon’s kitchen site.
Unleashing the Ideavirus 36 http://www.ideavirus.com
The End Of The Zero Sum Game
Traditional advertising is a game with winners and losers. If your product gets attention from
the targeted consumer, you win “mindshare” and your customer loses time. When a
consumer is foolish enough to listen to an irrelevant ad, she loses time and doesn’t even gain
useful information. It’s an old economy model in which every transaction has someone
taking something.
Permission marketing and the ideavirus are both very different from this model. These
models create a game in which everyone can win! If there’s a great idea, and it moves through
the hive for free, everyone who touches it wins in several ways.
First, you as the consumer win for recommending it to a friend. This increases your status as
a powerful sneezer (or your compensation as a promiscuous sneezer.) Because you respect
your peers, you’re not suggesting or pitching something that doesn’t make your friends’ lives
better. Violate this respect and your power as a sneezer goes way down.
Definition: SNEEZER Some people are more likely to tell their friends about a great new
idea. These people are at the heart of the ideavirus. Identifying and courting sneezers is a key
success factor for ideamerchants.
Second, the recipient benefits as well. He benefits from the way the idea changes his life, and
he benefits because he now has the ability to sneeze the idea to someone else, thus increasing
his power.
Third, the creator of the idea succeeds because her idea propagates and because she can sell
souvenirs (speeches, consulting, value-added services) to people who are now open and
receptive to her idea.
My friend, Chris Meyer, co-author of Blur, had this to say: “The one thing that distinguishes
effective sneezing campaigns from ineffective ones is RESPECT for the time, attention, and
reputation of the next guy to catch the virus. It’s important to note that the decision to
sneeze is, in general, a distributed one, made by each of us as to whether to clog our friend’s
Unleashing the Ideavirus 37 http://www.ideavirus.com
email or whatever with the virus in question, because our (local, at least) reputation is at
stake.”
This insight goes to the core of why ideaviruses are succeeding and why traditional marketers
don’t immediately grasp this approach (or permission marketing for that matter.) The
distributed nature of the decision is the antithesis of the command-and-control General
Patton approach that marketers have taken previously.
The reason that The Red Herring, The Industry Standard and other magazines are jammed
with ads is not because the ads always work. They don’t. The reason the ads are purchased is
that in exchange for money the marketer gets the illusion that they’re in charge of the
conversation, at least for a few seconds.
Bill Bernbach, the dean of American Advertising, was co-founder of DDB Advertising. He
died twenty years ago, but before he left us, he pointed the way to this “new” way of
marketing:
“You cannot sell a man who isn’t listening; word of mouth is the best
medium of all; and dullness won’t sell your product, but neither will
irrelevant brilliance.”
The answer, of course, is to give people a reason to listen and then create an infrastructure
that will amplify their ability to spread word of mouth. And core to both of those tasks is the
new respect that marketers need to show newly powerful consumers.
Unleashing the Ideavirus 38 http://www.ideavirus.com
SECTION 2: How To Unleash An Ideavirus
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 39 http://www.ideavirus.com
While It May Appear Accidental, ItÕs Possible To Dramatically Increase The
Chances Your Ideavirus Will Catch On And Spread.
This is the really cool part. Once you understand the fundamental elements behind the
propagation of an ideavirus, you can unleash your own.
Just because ideaviruses have usually spread through unknown means or accidental events
doesn’t mean that there isn’t a science to building and managing them.
You can invest in designing your product to make it virusworthy. Then if you understand
the eight elements of the ideavirus formula, you increase your chances of spreading your
ideavirus with every step along the way.
This can change the way you approach all of your marketing. If launching an ideavirus is the
most powerful thing you can do for a product and service, and there are steps you can take to
increase the likelihood that this will occur, you’ve got to try!
Unleashing the Ideavirus 40 http://www.ideavirus.com
The Heart Of The Ideavirus: Sneezers
SNEEZERS Some people are far more likely to spread an ideavirus than others. Malcolm
Gladwell (author of the brilliant book and ideavirus, The Tipping Point) calls this the Law of
the Few and breaks the key virus spreaders into three groups: Connectors, Mavens and
Salespeople. What’s critical in the analysis is understanding that some folks are dead ends,
while others will enable and amplify your ideavirus.
In his best example, Gladwell talks about the success of Paul Revere in warning us that the
British were coming. It turns out that a second man, William Dawes, went on a similar ride
the same night—but his was a total failure.
Why did Dawes fail where Revere succeeded? It’s because people knew Paul Revere. They
trusted him. He had credibility. And so when he said something, people were willing to
listen and believe. Revere was a sneezer. Dawes, a loner, tried hard but couldn’t get the idea
to become a virus.
Sneezers are at the core of any ideavirus. Sneezers are the ones who when they tell ten or
twenty or 100 people—people believe them.
Unleashing the Ideavirus 41 http://www.ideavirus.com
Sneezers Are So Important, We Need To Subdivide Them
There are two basic kinds of sneezers:
Promiscuous Sneezers
This is your uncle the insurance salesman. These are members of a hive who can be counted
on to try to “sell” their favorite ideavirus to almost anyone, almost any time.
1.
Promiscuous sneezers can be motivated by money or other inducements.
2.
Promiscuous sneezers are rarely held in high esteem as opinion leaders, but if they’re
promiscuous enough, they can be extremely effective.
DEFINITION: HIVE People are not one amorphous mass. We’re self-organized into
groups, or hives that have several things in common: a way to communicate among
ourselves; spoken or unspoken rules and standards; a common history; fashion leaders. Some
examples: Fraternity brothers at a college, orthodox Jews, readers of Fast Company,
Deadheads.
Many of the Net businesses that are now being organized around ideaviruses are targeting
this group (people who are willing to sell to their friends for personal gain). Companies like
Mercata, All Advantage and even Amazon are offering inducements to customers that
compensate them for spreading ideas to their friends and acquaintances in an attempt to
acquire new customers. As the value of creating ideaviruses increases, we’ll see more of this,
and we’ll also see more and more people becoming promiscous sneezers—basically, we’re
paying folks enough to corrupt them into spreading ideas in exchange for cash.
Powerful Sneezers
The hat business is near the end of an eighty-year downward spiral to total irrelevance. Each
year has brought worse news, with one manufacturer after another going out of business, and
most towns left with one (if they’re lucky) haberdasher.
In the midst of all this dismal news, about twenty years ago there was one bright spot.
Harrison Ford. With a bullwhip. Wearing a hat.
Unleashing the Ideavirus 42 http://www.ideavirus.com
Indiana Jones sold more hats for Stetson than any single person since the invention of the
Marlboro Man. Why? Because Ford has the influence to set style, because his appearance in a
movie wearing that hat coaxed millions of men who wanted to be like him into buying a hat.
The paradox of the powerful sneezer is that he can’t be bought. Every time a powerful
sneezer accepts a bribe in exchange for spreading a virus, his power decreases. When Bruce
Springsteen does ads in Japan, or Whoopi Goldberg shills for Flooz, they have less leverage as
powerful sneezers. The public knows that they can be motivated by more than just taste.
In fact, every time a powerful sneezer tries something new and introduces a new idea, she
takes a risk. If her followers reject the virus (for whatever reason), her ability to introduce
future viruses decreases. For this reason, it’s difficult to manipulate powerful sneezers, and
equally difficult to predict what might motivate them to adopt an ideavirus.
Here’s an analogy that demonstrates the difference between promiscuous sneezers and
powerful sneezers, and more important, explains how they might converge:
Anyone can buy an ad in the Pennysaver, or even write and insert a “special advertising
section” in some fancy magazine. The advantage of this kind of presentation, obviously, is
that it gives the marketer complete control over how the message appears and what it says.
Advertising is basically paid sneezing. And because the public realizes that that’s all it is, it
doesn’t have an awful lot of credibility. It still works, but it’s not as effective as real sneezing
from a powerful sneezer.
On the other hand, it’s up to the editor in chief of the New York Times to decide what
articles appear in the paper. No matter how much money a marketer spends (even though
spending a lot might get you noticed by the editorial staff), there’s no guarantee that an
article will appear—and no guarantee that if it does appear, it will say what you want it to
say.
Unleashing the Ideavirus 43 http://www.ideavirus.com
Enter the web. There are plenty of websites where the line between editorial content and
advertising is blurred, where sponsoring a website also gives you the right to say what you
want to say.…
So let’s imagine for a second that the New York Times embraced this shocking idea. That
they said, “Okay marketers, write your own articles! And pay us to run them!” Now there’d
be some ground rules. First, the marketer would specify how much they’d be willing to pay
to have a story featured. For example, a restaurant could decide it might be worth $10,000
for a feature on their new chef to appear in print.
Second, the Times would get final say over what was printed.
Obviously, a wholesale switch from powerful sneezer to promiscuous sneezer would decimate
the circulation base of the Times. If the Times accepted any article, regardless of credibility
and interest, just because the marketer was the highest bidder, it would totally destroy the
paper within a week.
But what if the Times realized that picking only the very best articles that were submitted
(maybe just a few a day) could ensure that people would still be delighted to read the paper?
What if the Times knew that for every 199 badly written restaurant fluff pieces, a great one
would show up? And what if the editor in chief had enough guts to pick just the great articles
and resist pressure to completely sell out?
Journalistic handwringing aside, this is already happening (not at the fabled Times, of
course), and it’s going to happen more. It’s happening on websites. It’s happening on
television (witness the CBS coverage of iWon.com awarding prizes—CBS owns a chunk of
iWon.) And far more interesting than this tortured analogy, it’s already happening with
people’s personal sneezing ethics.
A hundred years ago, there weren’t many opportunities for playwrights, actors and captains
of industry to sell out. Today, Whoopi Goldberg pitches Flooz, William Shatner pitches
Priceline and Gerald Ford is on the board of directors of several companies. In each case, the
celebrity is shifting from role of influential, powerful, can’t-be-bought-I’m-a-style-statesman
Unleashing the Ideavirus 44 http://www.ideavirus.com
to promiscuous sneezer, available for sale. William Shatner had lost his ability to set style
through his actions—he was past his prime as a powerful sneezer. So the segue to paid
sneezer made sense for his career. It would probably be a dumb move for Tom Cruise or Mel
Gibson, though.
After I left Yahoo!, I had many opportunities to serve on boards and do endorsements. I
chose not to. Why? Because I didn’t want to squander the powerful sneezing points I’d
earned by writing my last book. The one ad I did, I did for free. I’m still hearing about it.
Think about your own situation…. Have you ever signed up a friend for MCI’s Friends and
Family program? Or tried to get someone to use your Amazon affiliate links to buy books?
Or join with you to buy something at Mercata.com? In every case, you’re getting paid to
alter your behavior. That makes you more promiscuous and less powerful.
As the Net makes it easier to measure ideaviruses and motivate sneezers, we’re going to see
far more people become Promiscuous Sneezers, but, at the same time, the role of the
powerful sneezer will become ever more important. As available attention becomes ever more
precious, we’re going to be far more likely to listen to someone who’s spreading a virus for
non-personal gain.
Epinions.com is a fascinating model of the intersection between the powerful and the
promiscuous sneezers. Here’s a site where hundreds of thousands of people come to hear the
opinions of thousands of sneezers. Everything is reviewed, from books to dishwashers. And
the reviewers are clearly identified and constantly ranked. Promiscuous sneezers (who get
paid to do the reviews) suddenly become powerful! How? If a lot of people read and like
your reviews, your reviews carry more weight, regardless of your compensation scheme.
“Xyz” has posted more than 1,000 reviews and been read more than 100,000 times. She’s
compensated every time someone reads one of her reviews, so she certainly qualifies as
promiscous. She works hard to get others to read her reviews. But at the same time, she’s
developing a reputation as a powerful sneezer.
Unleashing the Ideavirus 45 http://www.ideavirus.com
Referrals.com is a business based around the idea of paying people to help with job searches.
Instead of just giving some headhunter the names of five friends who might be perfect for a
job (and having the headhunter collect a $30,000 fee if you’re right), referrals.com turns the
idea upside down. With their system, YOU send the job offer along to your friends, and if
they take the job, you get a check for $4,000.
If Referrals.com only attracts promiscuous sneezers, the business will fail. Why? Because the
very best people try hard not to listen to interruptions from promiscuous sneezers. The very
best people know that if someone can be bought, they’re not much more than a walking
billboard, and just as they ignore the billboards on the highway, they’re going to ignore the
most promiscuous sneezers in their midst. (Aside: If you’ve ever been called by a headhunter,
you know just how promiscuous people are willing to be in exchange for cash!)
Referrals.com is working very hard to turn powerful sneezers within very select, high-end
hives into people who, on occasion, are willing to sell out for a $4,000 payoff. These are folks
who might not hassle you just so they can make $5 or $10 in bonuses. But the idea of
becoming a headhunter and making $4,000 in exchange for sending a few emails is too
irresistible to pass up. This idea that even the powerful can become promiscuous for the right
inducement and in the right setting is a key building block to unleashing the ideavirus in an
organized way.
What a paradox. Powerful sneezers become less powerful when you buy them off. But
sometimes, promiscuous sneezers become powerful again when they get particularly
successful at it. It’s a cycle, with people switching off from one to another, always trying to
figure out how to be both promiscuous (read profitable) and powerful.
Unleashing the Ideavirus 46 http://www.ideavirus.com
The Art Of The Promiscuous
How do you attract and keep promiscuous sneezers? There are six key principles:
1. Make big promises
2. Show them how to make it up in volume
3. Describe an attainable path
4. When someone succeeds, tell the rest of them
5. Give the successful ones a way to show the non-sneezers it worked
6. Have a Mary Kay convention
Make big promises
One of the things that drives someone to become a promiscuous sneezer is the opportunity
for a change in lifestyle. Certain rewards, though small, are not as enticing as slightly less
certain rewards that are much larger. Human nature (especially among the optimists) will
give you the benefit of the doubt on the risks, but it won’t cut you any slack on the rewards.
So, I’m much more likely to help you out for a chance to get free dry cleaning for six months
than I am to get a certain reward of $4 off my next dry cleaning bill.
Show them how to make it up in volume
Of course, the promise has to be believable. One of the best ways to do that is to make it
clear to the promiscuous sneezer that the system can be gamed. That if they work the system,
the odds of winning go way up.
If I look at the offer you make and say, “Wait. If I go to ten friends, not just one, then I’m a
lock to win this great prize…” you’ve done it right. I may think I’m scamming you by going
to so many people to adjust the odds in my favor, but actually, I’m doing just what you
wanted me to do—and then some.
Many of the online affiliate programs work this way. These programs offer a commission for
referrals that result in a sale. First designed as a cheap way to get new customers referred from
relevant web sites, they’ve evolved into something far bigger. If you’re at an online pet store,
Unleashing the Ideavirus 47 http://www.ideavirus.com
for example, and you see a link to a book about training dogs, you can click on the link and
buy it from Amazon.com. Amazon then sends the affiliate (the online pet store) a
commission. Small businesspeople have looked at these programs and said “Wait! If I build a
site that does nothing but sell books and Barnes and Noble does all the work, I’ll scam the
system and make a ton of money.” Of course, the online bookstore doesn’t care a wit about
where the customers come from. They’re just happy to have them. In essence, hundreds of
thousands of entrepreneurs are now building businesses dedicated to finding customers for
other merchants.
Describe an attainable path
Alas, trust is in short supply, even among optimistic promiscuous sneezers. Thus you’ve got
to make it clear to potential sneezers that there is in fact a way for them to profit from this
adventure.
This is especially true for offers where you don’t have a lot of time to make your case. By
showing the sneezer how smooth the system is, by making it trivially easy to forward that
email or whisper to that friend, you’re far more likely to get their initial enthusiasm. The first
few sneezes are the most difficult to get an individual to perform.
When someone succeeds, tell the rest of them
This is so important and so overlooked. I’m presuming that you’ve gained permission to talk
with your sneezers on an ongoing basis. So now talk to them! I’m a member of several online
affiliate programs, but not one of them does this. Why not send announcements detailing
how the most effective affiliates are doing? Why not invite me to visit their sites and see them
in action? By making it really clear that some sneezers are happily profiting, you dramatically
increase the chances you’ll get better performance from the rest of your sneezers.
Give the successful ones a way to show the non-sneezers it worked
Mary Kay cosmetics gives its best salespeople a pink Cadillac. This is no accident.
There are plenty of ways to pay off a promiscuous sneezer. Why do it with a pink Cadillac?
Because it is a persistent amplifier of this sneezer’s success. Because it attracts new sneezers to
Unleashing the Ideavirus 48 http://www.ideavirus.com
the fold. Because it’s proof to the rest of your organization and to the world that you can get
rich by selling cosmetics to your friends.
Have a sales convention
Just because it’s a new century doesn’t mean we should abandon the idea of getting together
in real life. Zig Ziglar tells the story of how Mary Kay went to a sales convention when she
was a struggling salesperson. She didn’t even have enough money to eat the meals there…
she brought her own crackers and cheese. But at the final banquet, when the salespeople
queued up to shake the company president’s hand, Mary Kay looked at him and said, “Next
year, I’ll be back as the #1 salesperson.” The president, who could have easily brushed off the
claim, stopped what he was doing, paused for a full thirty seconds, looked her in the eye and
said, “Yes, yes, I believe you will.”
And the rest is sales history. But without the convention, I seriously doubt this would have
occurred. How can you get together with your best promiscuous sneezers?
In addition to these six principles, there are two things you can do to totally and completely
wreck your network of promiscuous sneezers:
1. Change the rules in the middle
2. View the relationship as an expense
Don’t change the rules in the middle
Alladvantage.com is one of the fastest growing websites on the planet. The idea was to create
a multi-level marketing organization where each member would get paid for the ads they saw
and, more importantly, for the ads seen by the people they recruited. This led to a classic
MLM (multi-level marketing) network marketing business, where people made more money
bringing in new salespeople than they did actually using the product.
After growing to more than five million registered users, the company took a look at the
numbers and realized that the path to profitability was going to be hampered by the high
rates they were paying. So, well within the fine print they had published when they first
started, they changed the rates.
Unleashing the Ideavirus 49 http://www.ideavirus.com
All hell broke loose. The very best sneezers started sneezing against the company. The growth
rate hiccupped. Bad news. They’ll survive, and they might even continue their record
growth. But far better to have run the numbers in advance and had a payment schedule they
could live with forever.
Don’t view the relationship as an expense
It’s so easy to move your relationship with promiscuous sneezers from investment to expense.
After all, at the beginning it’s great because these people are dramatically cutting your
acquisition costs and helping you grow. But once you do grow, it’s easy to assume your
growth might be able to continue without the “high cost” of paying your sneezers.
In practice, there are two terrible side effects. The first is that you’ll inevitably try to trim the
benefits you offer your sneezers as well as the effort you put into keeping them happy. Better
to just cancel the program outright than to start disappointing these critical allies (remember,
an unhappy promiscuous sneezer can quickly become an angry powerful sneezer).
Second, you’ll find yourself trying to grow using techniques that you haven’t evolved, tested,
measured or practiced. And more often than not, that means failure.
A better strategy is to put a cap on your new sneezer acquisition efforts at the same time you
love and reward your existing sneezers. During this interregnum period, get really good at
tapping other ways to grow. Only after you’re confident that you’ve got the transition
working should you start to phase out the sneezers who got you there in the first place.
Unleashing the Ideavirus 50 http://www.ideavirus.com
ItÕs More Than Just Word Of Mouth
Marketers have been pursuing word of mouth for years. There are five important principles
that someone unleashing an ideavirus should understand—principles that marketers
pursuing old-fashioned word of mouth didn’t use:
1. An idea merchant understands that creating the virus is the single most important part of
her job. So she’ll spend all her time and money on creating a product and environment that
feeds the virus.
2. An idea merchant understands that by manipulating the key elements of idea
propagation—the velocity, the vector, the smoothness, the persistence and the identification
of sneezers—she can dramatically alter a virus’s success.
Definition: PERSISTENCE Some ideas stick around a long time with each person,
influencing them (and those they sneeze on) for months or years to come. Others have a
much shorter half-life before they fade out.
Definition: VECTOR As an ideavirus moves through a population, it usually follows a
vector. It could be a movement toward a certain geographic or demographic audience, for
example. Sometimes an ideavirus starts in a sub-group and then breaks through that niche
into the public consciousness. Other times, it works its way through a group and then just
stops. Napster vectored straight to college kids. Why? Because they combined the three
things necessary for the virus to catch on: fast connection, spare time and an obsession with
new music.
3. The idea merchant remembers that digital word of mouth is a permanent written record
online, a legacy that will follow the product, for good or for ill, forever.
4. An idea merchant realizes that the primary goal of a product or service is not just to satisfy
the needs of one user. It has to deliver so much wow, be so cool, so neat and so productive
that the user tells five friends. Products market themselves by creating and reinforcing
ideaviruses.
5. An idea merchant knows that the ideavirus follows a lifecycle and decides at which
moment to shift from paying to spread it, to charging the user and profiting from it.
Unleashing the Ideavirus 51 http://www.ideavirus.com
An Ideavirus Adores A Vacuum
It’s very hard to keep two conflicting ideaviruses in your head at the same time
(Communism: evil or benign? Martha Stewart: pro or con? Can’t have both). So if an idea
already inhabits space in your consumer’s brain, your idea can’t peacefully coexist. It usually
has to dislodge a different idea, the incumbent, and that’s always tough.
Given that, the best friend of an ideavirus is a vacuum. When “60 Minutes” ran the story
about runaway acceleration in Audi cars, it was an ideal ideavirus. Why? Because most people
had never driven an Audi. Most people had never interacted with the Audi company. Most
people didn’t have a best friend who loved his Audi. As a result, the virus rushed in, filled the
vacuum and refused to be dislodged.
Audi, of course, did exactly the wrong thing in fighting the virus. They issued a tight-lipped
response and relied on engineering data to PROVE that they were right. Very correct, very
German and totally ineffective. It cost the company billions of dollars in lost sales.
Audi didn’t have to go out and spread the idea that Audi’s were good cars. That would have
been pretty straightforward if they were starting from scratch. Instead, Audi had to undo the
idea that had been spread by “60 Minutes”. And responding “did not” to TV’s “did too” was
a recipe for failure.
Instead, they could have countered the virus by filling in the rest of the vacuum. I would
have advised them to put an Audi 5000 in every major shopping mall in America. Let people
sit in it. Invite them to take the “Audi Sudden Acceleration Test” and see for themselves
what the car was like. By creating a more vivid and forceful alternative to a television hatchet
job, Audi could have unleashed its own countervirus.
At the beginning, the Internet was a vacuum. A Yahoo! or an eBay or an Amazon could walk
in and propagate its ideavirus fast and cheap. Today, though, launching a new search engine
or a new email service is hard indeed. Why? Because the vacuum’s gone.
Unleashing the Ideavirus 52 http://www.ideavirus.com
Take the much-coveted Aeron chair from Herman Miller. The company introduced this
puffy, bouncy desk chair for star executives and invented a market where none had
previously existed. Suddenly, you could spend a lot of money on a chair that actually worked
better, as opposed to just one that made you look bigger when you were busy firing people.
When Internet marketing pioneer Site Specific raised its first round of venture capital, the
principals went out and spent $15,000 on these chairs! This is a chair so remarkable, it was
featured on the front page of the Wall Street Journal.
Now, of course, there are plenty of neat, ergonomic desk chairs. One of Herman Miller’s
biggest competitors is betting the farm on their new Leap chair. Their MBA’s have taken a
hard look at Aeron’s success and market share and decided that they can capture x% of the
market. The problem, of course, is that there’s no longer a vacuum. The problem is that
now, instead of spreading a virus about how you can be more comfortable all day, they have
to spread a much smaller, and less compelling virus about why their chair is a little better
than the chair you’ve already heard of.
There are vacuums in your industry. But not for long….
Unleashing the Ideavirus 53 http://www.ideavirus.com
Once It Does Spread, An Ideavirus Follows A Lifecycle. Ignore The Lifecycle
And The Ideavirus Dies Out. Feed It Properly And You Can Ride It For A Long
Time.
Tom Peters co-wrote In Search of Excellence nearly twenty years ago. Through some smart
marketing moves (not to mention a great virus) the book became an epidemic and turned
into the bestselling business book ever written.
Tom’s career could have followed the arc of almost every other business writer… a big hit
followed by a long decline into obscurity. But instead of ignoring the lifecycle, Tom insisted
on riding it.
And he’s still riding it today. Every few years he unleashes a new ideavirus. He writes
mindblowing articles (like the “Brand Called You” cover piece for Fast Company a few years
ago) and follows up with books and exhausting worldwide speaking tours. When he shows
up in a town to give a speech, perhaps a third of the people there are dyed-in-the-wool Tom
Peters fans. And the rest of the audience? Brought there by the fans, exposed to his virus,
ready to be turned into fans.
By leveraging the base that his first book brought him, Tom has built a career out of
launching new ideaviruses. Sure, none of them were as big as In Search of Excellence, but the
vacuum keeps getting smaller, so the opportunities are smaller.
Other companies and ideas have ridden their first wave and then disappeared. People no
longer clamor to dance the Hustle or to get into Studio 54. They don’t visit the once hot
jennicam website or pay a premium for front row seats at Cats. Why? Because instead of
institutionalizing the process of improving, honing and launching new ideaviruses to replace
the dying ones, the “owners” of these viruses milked them until they died.
Unleashing the Ideavirus 54 http://www.ideavirus.com
Viral Marketing Is An Ideavirus, But Not All Ideaviruses Are Viral Marketing
Viral marketing is a special case of an ideavirus. Viral marketing is an ideavirus in which the
medium of the virus IS the product. It’s an idea where the idea is the amplifier.
DEFINITION: AMPLIFIER A key difference between word of mouth and an ideavirus is
that word of mouth dies out while an ideavirus gets bigger. Why? Because something
amplifies the recommendations to a far larger audience. That could be TV or other forms of
media (a good review in the New York Times that amplifies the message of one reviewer to
many readers) or it could be the web (a site like planetfeedback.com amplifies the message of
a single user).
Steve Jurvetson, the venture capitalist behind Hotmail, coined the term “viral marketing” to
describe the way the service grew. Hotmail offered free email. That alone was a very
compelling two-word business proposition. But the magic of the company was that in every
single email you sent using the service, there was a little ad on the bottom of the note. And
the ad said, “Get Your Private, Free Email from Hotmail at http://www.hotmail.com”.
Every time you sent a note, you spread the virus. The magic of viral marketing is that the
medium carries the message. The more you use Hotmail, the more you spread the virus. But
note: It was also extremely smooth…. The Hotmail site was just a click away from an email,
and it took just a few clicks more to start using it—and sending Hotmail’s built-in ads to
your friends.
Unfortunately, not every product lends itself to viral marketing. Viral marketing requires
that the product you’re using be communications-focused or very public. The new VW
Beetle is an example of viral marketing. Why? Because the more you drive it, the more
people see it. And the more Beetles people see, the more they want one. It’s not audible and
it’s not as smooth as Hotmail, but it is most definitely viral.
Many of the very best Internet ideas are built around some level of viral marketing. Using an
earlier example, Referrals.com pays big money to people who recruit their friends for hot
Unleashing the Ideavirus 55 http://www.ideavirus.com
jobs. Of course, the act of recruiting your friends is also the act of telling them about
Referrals.com.
Try not to get too obsessed with the magic, self-referencing nature of viral marketing
companies. They’re a very special case—for example, it’s hard to imagine how most books
could use viral marketing. Interesting, though, that line-dances like the Hustle and the
Macarena DID use viral marketing. After all, you can’t do the dance unless you teach your
friends how!
Unleashing the Ideavirus 56 http://www.ideavirus.com
What Does It Take To Build And Spread An Ideavirus?
There are two questions you can ask yourself about your idea before you launch
it…questions that will help you determine how likely your idea will become an ideavirus.
Is it worth it?
Nobody spreads an ideavirus as a favor to you. They do it because it’s remarkable, thoughtprovoking,
important, profitable, funny, horrible or beautiful. In today’s winner-take-all
world, there’s no room for a me-too offering, or worse, BORING products and services. If
it’s not compelling, it will never lead to an ideavirus.
Face it. Nobody is going to hand out big rewards ever again for being on time, performing
work of good quality, being useful, finishing a project on budget or being good enough.
That’s expected. That’s a given. The rewards (and the ideavirus) belong to the first, the
fastest, the coolest, the very best.
The biggest mistake companies make is that they chicken out. If your idea doesn’t become a
virus, it’s most likely because it didn’t deserve to become a virus.
If you’re now defining yourself as an idea merchant (hey, it’s either that or lose), then you
must accept the fact that being brave and bold in the creation of ideas is the only reason you
went to work today.
Is it smooth?
After someone’s been exposed to an ideavirus just once, they’re not likely to actually catch it.
We’ve made our brains bulletproof and ideaproof. There’s so much clutter, so much noise,
so many ideas to choose from that the vast majority of them fail to make a dent.
Think about the last time you walked through a bookstore (the home of ideaviruses waiting
to happen). How many books did you stop and look at? Pick up? Turn over? And how many
of those books ended up in your shopping basket? Got read? Led you to tell ten friends?
Precious few, that’s for sure.
Unleashing the Ideavirus 57 http://www.ideavirus.com
Compare this to the Harry Potter phenomenon… the bestselling books of the last few years,
created just because kids told kids. A classic ideavirus, and one that initially grew with no
promotion at all from the publisher.
It’s difficult to get from awareness to the “sale” of an idea, to convert a stranger into a friend
and a friend into a carrier of your ideavirus. An ideavirus succeeds when it pierces our natural
defenses and makes an impact.
In greek mythology, they tell the story of the Medusa. The Medusa was part of the race of
Gorgons—beings with a horrible curse. Anyone who looked in their eyes immediately and
permanently turned to stone.
There are plenty of marketers who wish that their ads or their product had the power of the
Medusa: that every person who saw it would be immediately transfixed, rooted to the spot,
and converted into a customer for life. (Of course, they don’t want their customers to die a
horrible death and be turned into stone, but I couldn’t find a Greek myth in which an evil
goddess turned you into a frequent shopper of Kate Spade purses, getting a second mortgage
just to pay for them.)
Alas, there are precious few Gorgon products and even fewer ad campaigns with Gorgon-like
properties. It’s foolish to expect that one exposure to your message will instantly convert
someone from stranger to raving ideavirus-spreading fan. So plan on a process. Plan on a
method that takes people from where they are to where you want them to go.
And while you’re at it, work on the product. Because a catchier, more compelling, more viral
product makes your job 100 times easier.
These are critical decisions because of the attention deficit marketers are facing. In 1986, the
year I published my first book, there were about 300 other business books published. In
1998, there were 1,778 business books brought to market.
The supermarket sees about 15,000 new products introduced every year. The Levenger
catalog alone features more than 50 different pens and pencils, none of which were available
Unleashing the Ideavirus 58 http://www.ideavirus.com
just a couple years ago. There isn’t a marketplace out there that isn’t more crowded than it
was a decade ago.
In a world where products are screaming for attention, the most precious commodity is
attention. And attention is harder and harder to achieve.
If you already understand the power of permission, your next question might be, “Fine, but
how do we get permission? How do we get the first date… the first interaction where we ask
people if we can start an ongoing dialogue about our products and their needs?”
My answer used to be a rather weak mumble about buying ads. The right answer, however,
is to create an ideavirus. The right answer is to let the market tell itself about your products
and services and give you permission to continue the dialogue without your having to pay for
it each time. The right answer is to create products so dynamic and virusworthy that you
earn the attention.
Unleashing the Ideavirus 59 http://www.ideavirus.com
There Are Three Key Levers That Determine How Your Ideavirus Will Spread:
Where do you start? What are the key elements worth focusing on to turbocharge your idea
and turn it into a virus? There are three things to focus on:
1. How big do you launch?
2. How smooth is it?
3. How can you turn trial into persistence?
1. How many people know about it before the spreading starts?
You can launch big or you can launch small. Vindigo (a viral phenomenon discussed in
detail later) launched their Palm ideavirus with just 100 people. Within weeks, that number
had grown to 3,000, and then quickly to more than 100,000. All without advertising.
However, if you’re entering a vacuum and there’s plenty of competition on the horizon,
launching big (while more expensive) can increase the chances that you’ll succeed.
How to launch big? With traditional interruption advertising. With sponsorships. With free
samples. One of the dumbest things marketers do is put artificial barriers in the way of trial.
For example, it’s obvious that one of the best ways to kill sales of a new car is to charge
people $100 to take a test drive.
But charging for a test drive is just as dumb as a politician charging people to hear a speech,
or a movie studio charging for the coming attractions. When you launch an ideavirus, the
more people who can see it fast, the faster it will spread.
Unleashing the Ideavirus 60 http://www.ideavirus.com
2. The importance of smoothness.
In addition to being persistent and cool, an ideavirus spreads the fastest when it’s smooth.
Persistence matters because the longer people are sneezing about your idea, the more people
they infect. Cool is critical because if it’s not virusworthy, it’s just not going to take off. But
smooth is essential because if you make it easy for the virus to spread, it’s more likely to do
so. In viral marketing (for products like the Polaroid camera and Ofoto.com) the ideal
solution is to build smooth transference tools right into the idea—which can be difficult.
But that doesn’t mean you shouldn’t try. Amazon tried with “Member Get a Member”
promotions, in which they bribe members to tell their friends to buy books from Amazon
(get $5 for your friends and $5 for you!). ZDNet puts a button next to every story they
publish on their website: click here to send this article to a friend. Smooth.
Tupperware built an entire company around the smooth transfer of product enthusiasm
from one friend to another. When you have a Tupperware party you are simultaneously
hanging out with friends, demonstrating products you like, selling them and recruiting other
Unleashing the Ideavirus 61 http://www.ideavirus.com
people to do the same to their friends. By focusing obsessively on how to make it smooth,
you can dramatically increase the velocity of the ideavirus.
3. Turning trial into persistence.
Sooner or later, you’ve got to turn momentary attention into an embrace of your idea, and
then, hopefully, into conversion of the user into a sneezer.
Permission marketing becomes a critical tool in working people through this transition. The
Hare Krishnas have grown their sect by inviting people to eat a vegetarian dinner with them.
Intrigued or just hungry, people give them momentary attention and then permission to talk
to them about this new way of life.
Sometimes people leave, having done nothing but eaten dinner. Sometimes, people listen to
what’s being said and decide to embrace the ideals being discussed. And sometimes, they
become converted and turn into sneezers, volunteering to go out and invite other people over
for dinner the next night.
Note that they didn’t start by walking up to a stranger and proselytizing about their religion.
Instead, they used a gradual technique to sell their idea effectively and turn it into a virus.
Are there religions that are not viruses? Sure, the Shakers were. They didn’t try to convert at
all. That’s why there are no Shakers left.
On the web, this multi-step process is too often overlooked by companies facing short-term
financial pressure (combine this with the legendary short attention span of entrepreneurs and
you can see why this happens). Instead of building a virusworthy cool product or service,
identifying a hive, promoting an idea, and making it smooth and persistent, they just spend a
few million dollars to buy advertising.
The hope, of course, is that somehow by spending enough money on clever ads, they’ll
magically create a critical mass of positive energy that will turn their idea into a virus.
They’re looking for a shortcut, and as a result, leading their companies to doom. Building a
virus takes insight, talent and most of all, patience.
Unleashing the Ideavirus 62 http://www.ideavirus.com
After a consumer is interested enough to visit ZDNet or Google.com or some other neat new
site, what should these sites do to augment the ideavirus? Three things:
1. Get permission to follow up: make it easy for me to learn about why I should embrace this
idea over time. All those ads you ran are a great way to get someone to your site, but it might
cost your site $100 in marketing expenditures to get that one visit from just one consumer. If
you don’t get permission to follow up, the entire $100 is wasted.
2. Make as many supporting manifestos available as possible, in whatever forms necessary, to
turn consumers from skeptics into converts. This can include endorsements, press reviews,
even criticisms and commonly made objections. Think of the Hare Krishnas at dinner. The
more they can expose you to during that hour, the better the odds of spreading the virus.
3. Make it easy for consumers to spread the ideavirus by providing a multitude of tell-afriend
tools, as well as overt rewards for becoming a sneezer.
Unleashing the Ideavirus 63 http://www.ideavirus.com
Ten Questions Ideavirus Marketers Want Answered
1.
Have we chosen a hive we’re capable of dominating?
2.
How likely are the powerful sneezers to adopt our virus?
3.
Do we know who the powerful sneezers are and how to contact them?
4.
What can we do to our product to make it more virusworthy?
5.
Are we rewarding promiscuous sneezers sufficiently to get them on our side?
6.
Have we figured out what we want the sneezers to say? How are we teaching them to say
it?
7.
Even if our product isn’t purely viral by nature, is it possible to add more viral marketing
elements to it (or to our marketing approach)?
8.
Do we know how to get permission from people once they’ve been touched by the virus?
Do we know what to say after we get permission?
9.
How smooth is the transfer of the ideavirus?
10. Is our offering good enough to wow this hive?
11. Do we have the resources and time to dominate this hive before others rush in to fill the
vacuum?
12. Have we built in multiple feedback loops so we can alter the virus as it moves and grows?
13. Have we identified the vector we want the virus to move in, and have we built the tools
and plans to keep it moving in the vector we’d like?
Unleashing the Ideavirus 64 http://www.ideavirus.com
Five Ways To Unleash An Ideavirus
Of the five ways to unleash an ideavirus, the most important element they share is that for
best results you must build this thinking in from the very beginning. If you’ve got an existing
product or service and you’re hoping to build a virus around it, your job will be more
difficult. The ideas behind the lightning fast success stories have all worked because the
ideavirus concept was baked in from the start. That’s one of the reasons more established
companies are having so much trouble competing in the new economy—they’re restricted
because of the standards and systems they built in years ago.
The five techniques, in order of sheer market power, are:
1. Go full viral. The more you use it, the more you market it (whether you want to or not).
In essence, using the product is the same as marketing it.
2. Pay off the promiscuous.
3. Make it smooth for the powerful.
4. Digitally augment word of mouth.
5. Altruism…reward the friends of the promiscuous.
1. Go full viral. This is the holy grail of ideavirus marketing. The beauty of viral marketing is
that if you properly construct the virus, you can grow like a weed and dominate the
market—if you can do it before the competition.
Polaroid and Hotmail are the poster children for viral marketing, but there are a few other
that are worth looking at:
Blue Mountain Arts was a pioneer in creating a virus around the idea of sending electronic
greeting cards. The virus is simple to understand—in order to send a greeting card
Unleashing the Ideavirus 65 http://www.ideavirus.com
successfully, you’ve got to send it to someone. Of course, once someone receives the card, if
they like the idea, they’re just a click away from sending someone else a card!
Even though the cards featured by Blue Mountain Arts could charitably be called “cheesy,”
the virus caught on. People got the idea that it might be fun to send electronic cards to their
friends… and the idea spread. The company started small, with no real advertising. Just a
few people sent the first batch of cards.
But then the magic of viral marketing kicked in. Let’s assume that each person sends cards to
five people. Let’s also assume that those recipients have a 50% chance of being interested
enough in the concept to go to the site and send cards to five of their friends. If we start with
ten people, the generations look like this:
10 people send 50 cards
which means that 25 people get the virus and send 125 cards
which means that 63 people get the virus and send 315 cards
which means that 162 people get the virus and send 810 cards
which means that 405 people get the virus and send 2025 cards…
Now, that may seem like a slow start, but if you assume that each generation takes three days
to occur (I send out ten cards and within three days, five friends show up and do the same
thing), then you’d have 58 million users in 54 days!
Of course, that doesn’t really happen. It’s unlikely you’ll be able to continue to get a 50%
conversion rate. And it’s certain that you’ll soon hit duplication, with individuals starting to
get cards from different people. But the math is nevertheless stunning.
The key number in the equation is the percentage of people who convert. If you lower it
from 50% in the Blue Mountain Arts example to 30%, the number of users drops from 58
million to less than 10,000. Which is why conversion is so critical.
The battle between Hallmark and Blue Mountain in this space is fascinating. Hallmark and
American Greetings, both old-line card companies, were well aware of the potential of the
Unleashing the Ideavirus 66 http://www.ideavirus.com
Internet. But they were also unable to imagine a world in which cards didn’t cost money—so
they made the cards they sold online available for a fee.
As a result, no virus emerged from the Hallmark site. If someone was charmed by a card and
came to the site to send a few, they discovered that they’d have to pay to do that. They didn’t
convert. Conversion fell below the magic number and the virus never ignited.
You can compute the magic number by multiplying the number of cards the average user
sends (in the example above, it’s 5) by the percentage of people who convert (50%). In this
case, the magic number is 2.5, which is how much bigger each generation will be than the
one before. Until the magic number exceeds 1.2 or 1.3, it’s hard for a product to get viral fast
enough to beat the competition.
By focusing on smoothness (it’s only three clicks to send a card and it’s free, so go ahead and
try it), Blue Mountain built an amazing conversion machine. As a result, the site grew and
grew until Excite bought it for nearly a billion dollars worth of stock. Whatever Blue
Mountain’s goal—to make a lot of money, to affect a lot of people or to spread their idea far
and wide—they’ve succeeded.
Hallmark and American Greetings have seen the light, and now they, along with Yahoo! and
others, offer free greeting cards. The challenge that they face is that there’s no longer a
vacuum, so their ideavirus can’t spread as fast, and their magic number is far lower than that
which Blue Mountain Arts enjoyed at its peak (the number must go down as the population
of untouched people approaches zero).
Another example of viral marketing worth looking at is Ofoto. Ofoto is an Internet
alternative to Fotomat. Instead of dropping your film off at the corner, you send your digital
camera files to Ofoto and they send back beautiful prints.
This is a compelling story, but there isn’t enough money in the world to communicate it
through traditional marketing means. Kodak spends $100 million a year in advertising (and
has been advertising for a hundred years). On top of the huge amount of noise out there,
Unleashing the Ideavirus 67 http://www.ideavirus.com
there are just no easy media channels Ofoto can use to spread its message in a cost-effective,
fast way to the target hive: digital photography users.
So Ofoto also launched a digital photo album. This album lets you post your favorite digital
photos online, for free, and invite friends to come see them. Here’s the good part: a digital
photo album with no one looking at it is worthless!
Thus, once you upload your photos, you’ve got to motivate your friends and relatives to stop
by and see the photos. You become Ofoto’s #1 marketing weapon.
Take pictures of your kid’s soccer team. Upload them. Tell everyone on the team where to
find the photos.
Some of the parents will like the photos so much they’ll click a button and buy a print.
Ofoto has a new customer. Interestingly, the content was created by someone else —not the
person who bought the photo. This is an effect that never happens to Kodak.
Even better, some people who see the photos of the soccer team will realize that they too
would like to be able to post pictures for friends. So the torch passes, and Ofoto has added
another photographer to its ever growing stable.
It’s worth noting that the conversion rate for Ofoto is almost certainly going to be lower
than it was for Blue Mountain Arts. First, it’s much less smooth. In order to spread the word
that you’ve posted someone’s picture, you’ve got to find that person and tell them about it,
and then they’ve got to hustle themselves to a computer and go look at it… not as clean as
the all-electronic approach of Blue Mountain.
Second, the virus is less smooth. If I want to buy a print, I’ve got to enter my name and
address, AND I’ve got to pay for it. If I want to upload photos, I’ve got to figure out how to
use my digital camera upload files, or I’ve got to mail in my traditional film to Ofoto for
developing.
Unleashing the Ideavirus 68 http://www.ideavirus.com
Despite these obstacles, Ofoto has a very positive magic number as demonstrated by the fact
that they’ve amassed more than 500,000 users in less than 12 weeks.
The astute reader has probably noticed a critical difference between Hotmail and Blue
Mountain Arts vs. Ofoto.
Hotmail and Blue Mountain Arts are self-referencing ideaviruses. The virus spreads with the
use of the product whether the user wants it to or not. When you first start using Hotmail, the
self-promoting signature line promoting Hotmail is automatically included in every email
you send. You didn’t choose to do that (though you can turn it off), it just goes along
anyway.
In the case of Blue Mountain, the symbiotic relationship between the product and the
marketing is even more obvious. The card is the marketing, so using it is, by definition,
promoting it.
Ofoto, on the other hand, does no such thing. You could quite happily use Ofoto for
developing, sorting and storing your photos and never recommend it to anyone.
Clearly, if the marketing element is benign and totally integrated into your offering, your
magic number is going to be much higher; the symbiosis pays off with big dividends. The
product has 100% efficiency…every user becomes a promoter. The challenge is this: it only
works for a very select group of products and services—probably not yours.
Why have I gone to great lengths to point out that viral marketing is merely a subset of
ideavirus marketing? Because while very few of us will ever be lucky enough to enjoy the full
fruits of a viral marketing campaign, most of us can unleash an ideavirus.
2. Pay off the promiscuous.
Paying powerful sneezers in an effort to make them promiscuous (but have them keep the
power) is an extremely difficult balancing act, but if you can do it successfully, you can turn
it into a billion dollar business.
Unleashing the Ideavirus 69 http://www.ideavirus.com
Some people call it network marketing or multi-level marketing. Others think of it as a paid
celebrity endorsement. But it can be as simple as member-get-a-member for your local health
club.
The basic idea is simple: If your recommendation is going to help my business, I’m happy to
pay you to recommend me.
The implementations vary all over the map. When Nike paid the coach of the Duke
University basketball team millions of dollars (for him, not Duke) to coerce his team
members to switch to Nike shoes, they were turning a formerly powerful sneezer into a
promiscuous one. Why? When people see what the Blue Devils wear, they might decide to
wear the same thing.
On the Net, technology makes it easy to take this model and make it much more personal.
Amazon’s affiliate program, in which Amazon pays users a portion of the book revenue they
generate through referrals, is built around this model.
Go to http://www.permission.com. There, at the bottom of the page, is a link where you can buy a
copy of Permission Marketing. Click on it and it will take you to Barnes & Noble or
Amazon—right to the page on the site that sells Permission Marketing. Both stores give me a
kickback on every sale.
Did I send you to Amazon just because I’m going to get a kickback? Nope. It doesn’t do me
any good to recommend a bookseller where you won’t end up buying the book—I’ll end up
with no kickback and no book sales either. I recommended Amazon because you’re likely to
have one-click shopping already set up, increasing the chances the book will get sold. I also
recommended Barnes & Noble, because their affiliate program is at least as good, and some
of my customers would prefer to shop there. But the kickback still influenced my decision,
and has clearly motivated hundreds of thousands of individuals and businesses to set up links
to their favorite books at Amazon and at Barnes & Noble.
Unleashing the Ideavirus 70 http://www.ideavirus.com
This approach is far less risky than Nike’s. Nike has no idea if the Blue Devils actually sell
shoes. They also have to pay for the endorsement in advance, with no refunds if they’re
wrong.
Amazon and other affiliate marketers, on the other hand, are using the power of the Net to
create a deal with no losers and no downside. You can set up an affiliate link in a few
minutes. For free. If it works, you get paid. If it doesn’t work, you don’t. And it doesn’t cost
Amazon a dime.
Because of this risk model, affiliate programs are flourishing. Be Free, a leading provider of
services to marketers using this approach, calls it Performance Marketing. They currently list
235 websites that are offering affiliate programs.
While it may be interesting to earn a dollar or two on a sale (interesting, that is, if you can
sell thousands a month), some companies are taking a different tack.
Woody Chin, founder of Referrals.com, thinks he’s found a way to change the way people
interact when it comes to job hunts and other sorts of business-to-business commerce.
Instead of paying people a nickel or even a buck, he’s paying people $1,000 to $5,000 each
for that priceless commodity: a referral.
Here’s how job filling works before Referrals.com: Hire a contingency headhunter. Offer to
pay a third of the final salary, but only if you hire someone the headhunter brings along. So
the hunter stands to earn $20,000 or more.
Now, the headhunter hits the phones. She calls everyone she can, and basically begs for leads.
There’s no obvious benefit to the referrer, except for the possible goodwill that occurs when
you find a friend a job.
Woody and Referrals.com are aiming to change that.
With Referrals.com, the hiring manager sends out a description of the job to people she
thinks might know good candidates. These referrers can be people she knows in the industry,
Unleashing the Ideavirus 71 http://www.ideavirus.com
company insiders or super-agents (and anyone can be a super-agent—read on). The key here
is that the referrals are from people whose opinion she values. The description includes a
bounty she is willing to pay for a hire as well as a limit to how deep and how wide a referral
tree she desires.
It’s fascinating to see that Referrals.com is building in a limit to the ideavirus! They don’t
want any given job search to get out of control and start being passed from friend to friend
ad infinitum. Instead, they artificially limit how deep a job search can go into the
community. This limit ensures that employers can focus their searches on a certain hive
without it running amok throughout the entire population. The web has turned what might
have been a multi-level marketing business into a carefully regulated ideavirus.
Anyone who gets involved in referring can sign-up to be a “super-agent.” Once you sign up
as a super-agent, your performance ratings will be available to hiring managers (in recruiting)
looking to find experts to help with their search. And of course, you get first crack at the new
job listings.
Let’s say the company wants a CTO. Let’s say they’re willing to pay $5,000 for a successful
hire. And let’s say they’re only willing to go two levels down the referral tree.
Now, a super-agent can send an email to five people he knows who might be perfect for the
job. If one of them takes the job, the super-agent gets $5,000 just for sending five emails.
But let’s say none of the recipients want the job. But one of them knows someone who does.
Bang. He forwards the mail a second time, and this time it lands on the desk of the perfect
hire. Assuming this guy gets the job, the first super-agent and the second referrer split the
money.
All of a sudden, you’ve monetized word of mouth! Referrals.com could create a class of
thousands of “super-agents” who spend their time doing nothing but finding people through
networking. Essentially, it lets just about anyone become a contingency headhunter. (Now, I
know what you think of contingency headhunters… but the small scale of each person’s tree
makes it unlikely it’ll ever get that bad!)
Unleashing the Ideavirus 72 http://www.ideavirus.com
Of course, it goes deeper than this. If it works for headhunting, maybe it works for finding
new clients for Viant, or for people who are looking to take a cruise. Or what about real
estate? If everyone could become a contingency broker, doesn’t life online get interesting? If
the Internet succeeds when it monetizes previously random analog events (like garage sales at
eBay) then this may just be the killer app for this space.
Does Referrals.com work? I actually have no idea. It’s just launching. We don’t know if the
promiscuous will overwhelm the powerful and pollute the whole system. We don’t know the
velocity of the idea or how long this particular virus will last. But it’s clear that something will
replace the current model of headhunters spamming powerful sneezers and essentially
stealing their rolodex.
Alladvantage.com wanted to take the multi-level marketing approach instead. Each person
they signed up got a commission on the revenue generated by the people those people signed
up. And so on.
They got off to a very hot start, signing up millions of users in a very short period of time.
But now, according to the Wall Street Journal, they’ve discovered that maybe they were
paying these promiscuous sneezers too much to make any money in the end. So Alladvantage
just announced new rules in the way they pay their sneezers.
The result was predictable… their most important sneezers were outraged. When you pay
people to refer on your behalf, you’ve got to expect that they are indeed motivated by
money, and when the money goes, so will your sneezers.
Multi-level marketing has gotten a bad reputation among powerful sneezers. Why? Because
individuals are encouraged to suspend their judgment and embrace the idea that several
generations down the pike, they’ll be rich.
While this is a fine choice for an individual to make, it’s problematic for those who are
friends with this individual. Why? Because the personal interaction is no longer on a level
playing field. Person A uses his friendship with person B to encourage her to buy or use
Unleashing the Ideavirus 73 http://www.ideavirus.com
something that isn’t necessarily in her best interest. If she agrees, then person A sees a
significant return, while person B inevitably sees LESS of a return. If she resists, the
friendship is strained.
If the pyramid is steep enough (if there’s enough money promised at the end of the tunnel),
this sort of approach can work. But it usually leaves scorched earth in its path, and
disappointments in the form of broken friendships or financial promises not reached.
To date, very few companies—online or off—have figured out a way to turn network or
multi-level marketing into a large, sustainable business. Those that have, like Rexall, Amway
and perhaps Alladvantage, now have to work even harder to undo the bad reputation that
this approach has earned.
3. Make it smooth for the powerful.
One of the most elegant ways to take advantage of the new tight networking among
consumers is to identify the powerful members of a hive and make it as easy as possible for
them to tell each other about an ideavirus.
When online content sites first debuted, they were extremely hesitant about sharing their
articles. Some of them went so far as to make it impossible to copy and paste the text in an
article. They were petrified that one person would copy an article and no one else would
come to the site and see the ads.
What they soon learned, however, was that the easier they made it to share, the more likely
people were to tell their friends. And if someone came in to read one article, they were likely
to read more. ZDNet.com was one of the first sites I encountered that used this technique.
In one promotion my former company Yoyodyne did for them, they found that more than
20% of the people exposed to a compelling piece of content actually forwarded it to a friend.
Fast Company magazine—devoted to bootstrapping start-ups—does the same thing. Visit
http://www.fastcompany.com/team/wtaylor.html and you can see a list of the articles that co-
Unleashing the Ideavirus 74 http://www.ideavirus.com
founder Bill Taylor has written for that magazine. They’re all there, unabridged, and you can
read them for free.
But the smooth part of this wannabe ideavirus is the little button on the bottom that says
“Click here to send this page to a friend.” All you have to do is type in their email address
and your email address and—boom—it’s done. If his articles contain ideas that are
virusworthy, the Fast Company site is doing a good job in helping them go viral.
Inside.com, which sells subscriptions to its online media newsletter and website for $200, is
happy to have people send these pricey articles to non-subscribing friends. In fact, there’s a
big “send to a friend” button on the bottom of every article. The reason is obvious. Once
you’ve read one, you might be willing to pay for more. All they need is a few of the ideas
they publish to become viral and suddenly the business of selling subscriptions will get a lot
healthier.
In essence, Inside.com is hoping that its readers will market the site for them, spreading ideas
that might go viral and then bringing in new paying customers as a result.
4. Digitally augment word of mouth.
This is a really interesting way of looking at the fundamental change that’s occurring, and
understanding how word of mouth is different from an ideavirus.
If I was delighted by a movie in the old days, I’d tell a friend or two. My comments would
end up influencing three or four or six people.
There are plenty of books on this topic and marketers have always been enamored by the
potential of word of mouth. Alas, without amplification, it usually peters out.
Today, if I like a movie, I can post my comments on a variety of online movie sites. Or I can
email ten friends (who can each forward the mail to ten friends). Later, when the video
comes out, I can post my review on Amazon, where hundreds or thousands of people might
read it.
Unleashing the Ideavirus 75 http://www.ideavirus.com
Using a service like Epinions.com, I can go online and search out opinions on everything
from BMW motorcycles to summer camps.
What’s neat about digital word of mouth (let’s call it word of mouse) is:
1.
It is extremely persistent. Unlike a comment at the watercooler or over the phone, a
comment in a newsgroup, on Epionions or Amazon lasts forever.
2.
It has much greater velocity. The number of ripples my stone makes when dropped in
the pond of public opinion is far greater online. Why? Because if I tell you I like my car,
it might be months before that sort of car comes up again in conversation. But online,
conversations are happening 24 hours a day, and the “conversation” on any given web
page is precisely about what that page is about. As a result, the number of interactions
multiplies geometrically.
3.
It can have more credibility. At first, the opposite was true. An anonymous stock tip or
other form of online recommendation was totally suspect. The sneezer could be a paid
mole, or worse, someone with horrible taste. But now, thanks to rating systems and the
presence of powerful sneezers, it’s possible to know how congruent your tastes are with
those of the sneezer, so it ends up having a ton of credibility.
Amazon is now rating the reviewers! A visit to
http://www.amazon.com/exec/obidos/tg/cm/member-reviews/-/AFVQZQ8PW0L/102-72353452994554
shows me that Harriet Klausner is the top ranked reviewer on the entire site.
Harriet, a retired librarian, has written more than 500 reviews and has received more than
5,000 votes from other folks who agree with her taste. If Harriet likes a book that you like,
you’re certainly going to give her sneeze some credence in the future.
5. Altruism.
Several years ago, a hot chef in Chicago decided to go out on his own and open his first
restaurant. Realizing how competitive the market was, he did a neat thing. He never opened
it to the public. He refused to accept reservations from strangers.
Unleashing the Ideavirus 76 http://www.ideavirus.com
If you wanted to get into Les Nomades, you had to be a member. And how did you do that?
Well, the first 500 people were given memberships because the chef knew them as regular
customers at his old job, and he personally invited them.
Then he told each member that they were welcome to sponsor other members. All they had
to do was vouch for someone and he’d make them a member too.
So, what’s in it for the member to nominate someone else? Simple. They scored points with
their friends as powerful sneezers because they could “get you in” to the hottest restaurant in
town.
Of course, this wouldn’t have worked if the restaurant hadn’t been spectacular. But it was.
And it was exclusive. But by allowing his members to do his marketing for him, by giving
them an altruistic tool that increased their power as professional sneezers, the chef was able to
get out of the way and let his customers sell for him.
Unleashing the Ideavirus 77 http://www.ideavirus.com
SECTION THREE: The Ideavirus Formula
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 78 http://www.ideavirus.com
Managing Digitally-Augmented Word Of Mouth
That’s what I would have called this book if it had been published by the Harvard Business
Review. And you probably wouldn’t be reading it now! Words matter. Understanding
exactly what we’re talking about makes it far easier to actually do something about the world
around us. That’s why I take such great pains to invent new words and get us all thinking
about exactly what they mean.
If we bump into each other at some convention and you ask me to talk about your business,
I’ll instantly start using words like hive and sneezer and velocity and smoothness. Why?
Because these shorthand phrases make it easy for us to communicate. By using words that
indicate we both understand the underlying factors that leverage an ideavirus, we’re far
likelier to actually get something done.
The ideavirus formula has eight co-efficients. Each one represents not just a concept, but a
variable that you can tweak to make your product or service more viral, to create the
elements you need to drive your idea into the community.
Unleashing the Ideavirus 79 http://www.ideavirus.com
Tweak The Formula And Make It Work
It may be possible to write down the key elements of building and spreading a virus as a
mathematical formula. No, I don’t think you’ll use it. But understanding the co-efficients
makes it far easier to see what’s important and what’s not. They also help you see the wide
range of factors that can help an idea go viral; focusing on the most highly leveraged factor
for your idea is a first step in launching the virus.
Multiply these five factors:
[reputation benefit to powerful sneezer of recommending virus]
[selfish benefit to promiscuous sneezer of recommending virus]
[smoothness of sharing the virus with a friend]
[power of the amplifier used to spread positive word of mouth]
[frequency of interactions among hive members]
Divided by the sum of these two factors:
[number of times you need to expose someone in this hive in order for the virus to catch]
[number of different sneezers who have to recommend a virus to a given individual for it to
ignite]
And then multiply that by the product of these four factors:
[percentage of infected hive members likely to sneeze]
[number of people the infected sneezer is likely to contact]
[persistence of the virus (how long does a sneezer sneeze?)]
[number of people infected /(divided by) number of people in the hive]
Comments on each component:
[reputation benefit to powerful sneezer of recommending virus]
Powerful sneezers can’t be bought. But don’t forget that they are selfishly motivated. Will
this make me look smart? Will it make someone else happy? Will it make the world a better
place? There are plenty of levers that motivate powerful sneezers to spread the word, and they
are often complicated and subtle. Some of our favorite powerful sneezers: Zagats, Linus
Unleashing the Ideavirus 80 http://www.ideavirus.com
Torvald, Paul Newman, Ruth Reichl, Randall Rothenberg, Andy Hertzfeld, Chuck Close,
Spike Lee, Bill Taylor, Don Peppers, Peter Mayles, Alan Greenspan and Yo-Yo Ma. You may
not know all of these names, and there are plenty of hive-based sneezers I’ve never heard of,
but what they all have in common is that they’re perceived as insightful and altruistic. Once
people think they can be bought off, their power plummets.
[selfish benefit to promiscuous sneezer of recommending virus]
As we saw in the Amazon affiliate example, if you can make the benefit to the individual
both significant and easy to achieve, people will respond to it. Amazon signed up hundreds
of thousands of affiliates with a simple offer (get a percentage kickback on everything you
recommend) and backed it up with a two-minute procedure for qualifying and actually
getting started.
[smoothness of sharing the virus with a friend]
Once I want to tell someone about your idea, how do I do it? If it’s got a dumb, hard-to-say
name, or an embarrassing implication, I’ll probably pass. On the other hand, Hotmail is
smooth indeed, because every time I send email I’m talking about the idea.
The Polaroid camera used this smoothness brilliantly. After all, the only reason to take a
picture is to show it to other people, and if you can make the showing (and the waiting) turn
into a discussion of the idea, so much the better.
The beauty of Vindigo is similar. In order to tell you about Vindigo, I’m going to pull my
Palm out of my pocket and show it to you. But once I show it to you, I’m only one button
away from actually giving it to you. The thing I want to show you is how easy it is to give
you, so the virus self-reinforces.
Ideally, you’ll figure out not only what a sneezer should say to someone when they talk about
your idea, you’ll also make it easy and automatic for them to do so.
[power of the amplifier used to spread positive word of mouth]
The mother of a friend of mine was runner up for Miss America in the early 1960s. I think
she lost to Anita Bryant. Alas, coming in second did very little for her career. Anita, on the
Unleashing the Ideavirus 81 http://www.ideavirus.com
other hand, made her fortune squeezing oranges. Point is that once she conquered that hive
of a few judges, the news was amplified far and wide. And the amplification (as per Zipf’s
law) gave her the foundation to create a career.
A challenge in tailoring your ideavirus is to make sure that when you do conquer an
individual or dominate a hive, the good news is amplified as far as possible, preferably at no
cost to you.
[frequency of interactions among hive members]
Some hives (like teenage girls) interact with each other far more frequently (and with much
more intensity) than others—like senior citizens. By understanding the frequency of hive
interaction and then trying to focus on moments of high interactivity, you can dramatically
increase the velocity of a virus.
Trade shows, for example, bring sneezers together for intense periods of information
exchange. By doing something as simple as handing out hats with your logo on them, you
make it more likely that you’ll reinforce your message during this critical time.
[number of times you need to expose someone in this hive in order for the virus to catch]
Some viruses are smooth indeed. See them once and you understand them. It only took one
exposure to the Macarena to get it. In general, the simpler the idea and the lower the risk, the
more likely someone is to get infected. Most of all, though, this variable is driven by how
viral the idea is to begin with. Meaning: is it cool, wonderful, important, dramatically better
and fun?
[number of different sneezers who have to recommend a virus for it to ignite]
Not all ideas have Medusa qualities. We usually need to hear from external sources before
we’re willing to buy into the new thing, especially for risky ideas. Bestseller lists for books
and other products are terrific, as are the sort of seal-of-approval validations that institutional
sneezers look for. “Hey, if it’s good enough for IBM…” say the more timid prospects.
Unleashing the Ideavirus 82 http://www.ideavirus.com
Bestseller lists are a stand-in for the number of recommendations you need to decide. A
bestseller list says, “There are 24,000 other people who liked this idea.” The reviews on
Amazon are another great example of this. When 50 people post a positive review, it counts
for something.
The alternative, which also works, is actually hearing from sneezers one by one. Some ideas
need only one sneezer to get you try it (like a restaurant) while others might need a hundred
(like switching over to using email or a Palm to run your business).
[percentage of infected hive members likely to sneeze]
Some hives are filled with sneezers. And some ideas make people sneeze more than others.
When John McCain tried to capture his party’s presidential nomination, he discovered an
entire population of people, previously dormant, who were so moved by his candor and
campaign finance message that they started sneezing on his behalf. Not accidentally, many of
these sneezers were in the media, carrying his message far and wide.
Another variable is your ability to increase the likelihood that people who don’t usually
sneeze decide that they’ll make an exception just for you. Focus on the time and place of
your introduction to the hive. Want your employees to spread an important new idea among
themselves? Don’t introduce it at the Friday afternoon beer blast, but rather make it a special
event. Give them the tools they need to spread the word. Reward them for doing so, or make
it clear how the virus will dramatically help your company. It’s not an afterthought—it’s the
core of your marketing campaign.
[number of people the infected sneezer is likely to contact]
This is an especially important metric for promiscuous sneezers. Once you’ve converted
people into spreading your message for their own personal gain, how can you incent them to
spread the word to a LOT of their friends? One way to do this is by offering increasing
returns to the sneezer—the more you bring us, the more we give you (but be careful not to
turn sneezers into spammers, who end up proselytizing strangers and causing a backlash).
Referrals.com aims to do this by turning their best sneezers into super-agents, giving them
better information and more money.
Unleashing the Ideavirus 83 http://www.ideavirus.com
The same reasoning is obviously a factor in choosing which members of the media to
contact. Saul Hansell at the New York Times has far more reach and influence than Jason
Snaggs at the Phoenix Register. Seems obvious, but what most marketers miss is the fact that a
very small number of powerful sneezers can have an impact far outside their perceived
influence. A reporter with the right readers could have far more sway over your virus than
someone with plenty of reach but little influence.
[persistence of the virus (how long does a sneezer sneeze?)]
A short-lived experience that leaves no lasting effects is hard to turn into a virus, especially if
it’s not a social event like pop music (does every generation after ours realize just how bad
their pop tunes are?). Tattoos, on the other hand, are extraordinarily persistent, so even
though they’re not very smooth, they continue to infect people for decades, making up what
they lack in impact with sheer stick-to-it-ness.
[number of people infected /(divided by) number of people in the hive]
This is about measuring hive dominance. If just a small percentage of people in your chosen
hive have been infected, you really have your work cut out for you. While you shouldn’t
compromise the essence of your idea in order to get a wide platform, you should be superwary
that you don’t start with too small a sample of too large a hive. It’s very easy for your
virus to fade before it catches on.
Unleashing the Ideavirus 84 http://www.ideavirus.com
Advanced Riffs On The Eight Variables You Can Tweak In Building Your Virus
In this section, we’ll take a look at each of the eight underlying variables in the ideavirus
formula, and try to get a handle on exactly how you can manipulate them for your product.
No two industries rely on the eight fundamental principles in precisely the same way. But
virtually every ideavirus I’ve ever seen uses some of these principles in an extraordinary way,
and just about every one could be improved if it expanded further into the other areas.
The Eight:
1. Sneezers
2. Hive
3. Velocity
4. Vector
5. Medium
6. Smoothness
7. Persistence
8. Amplifier
Unleashing the Ideavirus 85 http://www.ideavirus.com
Sneezers
As described earlier, there are two kinds of sneezers: Powerful and Promiscuous. While all
eight elements of the formula are critical, this is the area where many brand marketers have
the most control, and thus the most influence.
Choose your sneezers—don’t let them choose you. By focusing obsessively on who you’re
choosing to sneeze on your behalf, you build the foundation for your virus.
Powerful sneezers are certainly the most seductive, in that the right word from the right
sneezer can make all the difference to your virus. If David Letterman visits your diner on
television, or the New Yorker writes a twenty-page rave about your website, or if you win a
MacArthur Fellowship Grant, well, you’ve really made it.
Oprah Winfrey is quite possibly the most successful sneezer of our generation. She has
single-handledly turned more than a dozen books into national bestsellers. She has launched
a magazine that already has more than half a million subscribers. She can influence millions
of the most powerful consumers in America, just by uttering a few sentences.
It’s interesting to see how effectively Oprah and her brandmate Martha Stewart have
successfully monetized their position as powerful sneezers. If they trip and get perceived as
promiscuous sneezers, as sneezers for hire, their effectiveness is quite reduced. But if they can
maintain their position at the same time they sell books and magazines or sheets and towels,
they’ve effectively leveraged their fame.
But few of us are that lucky. Most times, you’re going to have to focus on powerful but less
influential sneezers—individuals or organizations that have something to gain by endorsing
your idea but aren’t so out there that they’re tagged as promiscous sneezers.
Some powerful sneezers are very prominent and thus very hard to reach. The challenge for
most marketers is to find the second tier of sneezer—the approachable, interested sneezer
who can do almost as much for you as Oprah or Martha, but with whom you have a far
greater chance of making an impact.
Unleashing the Ideavirus 86 http://www.ideavirus.com
The story of The Bridges of Madison County is a great example of this. Warner Books, the
publisher, realized that most other publishers were doing very little to market to the
independent bookstores, and that if he could court them and give them something to sell
that made them feel special, it would translate into a bestseller.
Of course, as soon as the legions of independent booksellers succeeded in turning Bridges
into a phenomenon, they were assaulted by dozens of other less imaginative publishers, all
trying to rush in and use the same strategy. Too late. It got cluttered. They got busy. No one
else ever repeated the focused, obvious success of this approach.
Remember, an ideavirus adores a vacuum, and Bridges filled that vacuum. As other book
marketers rushed in, no one was able ever again to persuade a critical mass of booksellers to
support just one book.
Does this mean Warner was doomed never to be able to repeat this process again? Is that all
there is—just one new gimmick after another? No! Instead, Warner needed to gain
permission from this critical sneezer audience and use that permission to promote the next
book and the next through a channel they were clever enough to build.
Unleashing the Ideavirus 87 http://www.ideavirus.com
Hive
Winning with an ideavirus begins with the choice of hive. And this choice is so important,
I’d suggest the following: choose your hive first, then build the idea.
Traditionally, marketers start with a problem, or a factory, and go from there. I’ve got a great
widget, and now I need a way to take it to market. Or, we’ve got this excess plant
capacity—let’s find a way to fill it. But that’s not what works today. Choose your market by
identifying a hive that has a problem and has the right concentration of sneezers, the right
amplified networking, the right high velocity of communication and, most of all, an
appropriate vacuum.
Success will come to marketers who attack small but intimate hives. Yes, Yahoo! and eBay hit
huge home runs, but they’re remarkable precisely because success across such a large hive is
rare indeed. We can learn a more relevant lesson from magazines.
Fast Company is one of the fastest-growing (and most profitable) magazines ever. Why? Well
it certainly helps that it’s a great magazine. It also helps that the Internet created a huge
demand for this sort of advertising space. But the real success came in the hive that the
editors selected.
Turns out there are hundreds of thousands of people in mid-sized to large companies who
are eager to do a great job, but feel frustrated at the slow pace and mind-numbing
bureaucracy they face every day. Until Fast Company, the members of this hive didn’t even
know there were others just like them. They didn’t have a tool they could use to reach their
peers.
Fast Company became the identity, the bible, the badge of honor for this new hive. It gave
them a way to communicate, to learn and to have confidence in themselves. By every
measure, the magazine was virusworthy.
Just about every reader of Fast Company became a powerful sneezer. With no compensation
and very little encouragement, they started signing up co-workers for subscriptions, Xeroxing
Unleashing the Ideavirus 88 http://www.ideavirus.com
page after page of the magazine and passing it around the office. The readers even created a
worldwide network of support groups, meeting in cities on a monthly basis, with no help at
all from the magazine.
Fast Company filled a vacuum. It got viral. It enchanted and connected with a huge legion of
powerful sneezers. All because the editors chose the right hive and created a virusworthy
product.
A few years later, Time Warner launched Real Simple magazine, inspired by the significant
sales of books about simple living. So they launched a magazine dedicated to simplifying our
lives. Obviously, it’s aimed at a very different hive than that of Fast Company. Alas, the
magazine is off to a slow start.
Why?
Because this hive isn’t the right one at the right time. Because there’s a real lack of aggressive
powerful sneezers. Because the hive doesn’t have a built-in forum for communicating with
each other (it’s not office-centric like Fast Company). As a result, the magazine is having a
much harder time going viral.
Choosing your hive
The Zagats Guide to New York City Restaurants is a fascinating document. According to
Zagats, the book is put together by 100,000 reviewers, who ate out an average of four times a
week, spending an average of $40 a person. Do the math. That’s more than $8,000 of mostly
after-tax money spent on eating out every year.
This very special hive of people shares a demographic but is by no means typical of the U.S.
population (which in itself is very different from the world at large). Trying to appeal to
everyone is almost sure to fail, for the simple reason that everyone wants something different!
The reason there isn’t one restaurant in Cincinnati or Indianapolis or Tallahassee that’s as
good as the Union Square Café in New York is not that the population can’t afford the tab.
Unleashing the Ideavirus 89 http://www.ideavirus.com
There’s certainly enough money in those towns to keep the seats filled in several restaurants
of this ilk. It’s simply that the hive that can afford these restaurants don’t have a high velocity
way to get the word out fast enough to keep the restaurateur happy. And it’s not clear that
they’d persist. In other words, eating in a New York-style fancy restaurant probably isn’t the
way these “out-of-town” hives choose to spend their time and money. Same’s thing true for a
New York hive that wouldn’t reward a French restaurant that might do just great in Paris.
All of which is a very fancy way of saying, “If the hive doesn’t want it, you picked the wrong
hive.”
Selecting a hive that respects the core value of your virus is a critical first step in laying the
foundation for promoting the idea. College students want something different from
gardeners, who are quite different from computer geeks. Targeting everyone is a sure path to
failure.
Of course, the real reason you want to pick the right hive is not because their values match
the benefits of your product. It’s because when you pick the right hive (and a small enough
hive) you have a chance of overwhelming it—of pumping so much positive juice into your
digital word of mouth that you really do dominate, that so many sneezers are recommending
you to the rest of the hive that the majority surrenders and the entire hive converts.
Once your idea starts coursing through a hive again and again and again, you’ll have a piling
on effect. People will want to be exposed to your idea just because everyone else in the hive
they respect is talking about it.
The mistake that’s so easy to make is to get greedy as you choose your hive, to say, “this
product is for everyone” or “anyone can benefit from this idea.” Well, there are seven billion
people on the planet, so it’s unlikely your comment is correct; even if it is, there’s little
chance that a virus would spread across a hive that big.
Far better to pick smaller hives and conquer them a few at a time. Far better to identify
consumers when they’re grouped in bunches (at a trade show, say, or geographically) and
then allow the concentrated essence of your virus spread to other hives.
Unleashing the Ideavirus 90 http://www.ideavirus.com
Coors did this with beer years ago. You could only get Coors in Colorado, then you could
only get it west of the Mississippi. By concentrating their marketing dollars, they addressed a
smaller hive. This enabled them to get a larger percentage of the hive to sample the product.
This core group then had a smooth way to spread the word, and it quickly conquered one
state after another.
Without any effort from the Coors people, the virus spread to the East Coast. Coors fielded
thousands of requests from disappointed drinkers who wanted to try this new beer they’d
heard about, but couldn’t.
Coors dominated a hive. Then they went national to try to fulfill the demand created when
their hive spread the word. Unfortunately, the new hive was so large, it turned out to be
difficult to satisfy and dominate.
Compare the powerful, nearly effortless spread of their idea with the challenges they face
today. As a national brand in a stagnant market, growth by any method is hard to come by.
They built their company on a unique virus, but they couldn’t continue to grow their
company the same way.
Unleashing the Ideavirus 91 http://www.ideavirus.com
Velocity
Napster is a worldwide file sharing database that lets Internet users share MP3 files. In
essence, you can listen to the digital record collection of millions of other people. The idea
behind Napster turned into a virus and grew like crazy. Why?
They hit college campuses—a hotbed of communication. A virus can spread across a campus
in a matter of hours. When a dear friend of mine went to Tufts in the late 1970s his
roommate started a rumor that Paul McCartney had died (this was before John Lennon’s
tragic death—they weren’t that callous). Within an hour, they started hearing the rumor
back—from friends of friends of friends who couldn’t precisely remember where or how
they’d heard it.
Napster was spread the same way. How? Because in addition to being on a college campus,
Napster lives on the Internet. So, instead of being word of mouth as in the Paul McCartney
example, it was digitally augmented word of mouth. On college campuses, everyone has
email, and email is both instantaneous and amplified. You can send an email to thirty or
forty friends as easily as you can write to one. So once a powerful sneezer had tried the
software and confirmed that it worked as advertised, the word spread fast.
Why is velocity so important? Remember, filling a vacuum is far easier than going second. If
the velocity of a virus isn’t fast enough, a competitor may leapfrog past you into a new hive
before you can get there, dominating as the “original” in that market.
This happened with beer, in which regional favorites have long survived the introduction of
nationwide refrigerated delivery. It even happened with the college entrance exams, in which
the ACT is favored in the Midwest, years after the SAT became the standard almost
everywhere else in the world. The only reason this happened is that the ACT got to the
Midwest first.
How does the Net change our economy so dramatically? Because it dramatically increases the
velocity of viruses in various hives. Where it used to take weeks or months for a contractor to
Unleashing the Ideavirus 92 http://www.ideavirus.com
talk with suppliers before building an office tower, he can now do it in just a day using the
Net.
This increase in velocity fundamentally changes the dynamic of a virus. Something
newsworthy might have 20 or 30 or 100 cycles of communications before the issue itself
becomes boring. In the days before the Net, if each cycle only touched one or two or three
people, the virus would die before it got old. Today, these cycles allow the virus to mutate
and evolve as it touches millions of people.
Unleashing the Ideavirus 93 http://www.ideavirus.com
Vector
Richard Dawkins, a brilliant evolutionary theorist, had his own word for the phenomenon
I’m calling ideaviruses: memes. He pointed out that a meme was like a living organism,
surviving not in the real world, but in our world of ideas.
Like a real organism, memes could live and die, and more important, they could evolve.
Every time a meme is passed from person to person, it gets touched, changed
and—sometimes—improved.
Once a meme has been passed around enough, it ceases to evolve as quickly and just becomes
a building block for future memes. Pop singers are experts at stringing together memes and
turning them into concise snapshots of our lives. (Paul Simon is a favorite—Graceland,
Kodachrome, the pop charts… you get the idea).
One of the behaviors noticed by Dawkins and practiced by anyone who markets with
ideaviruses is that memes follow a vector. An idea doesn’t spread evenly and nicely through a
population. Instead, people are more likely to send it in one direction instead of another.
At college, there was always someone who knew where the good parties were (and which
ones to avoid). In your town, there’s someone who just seems to have the inside buzz on
which restaurants are hot. On the Internet, some people seem to be on the vector of the
latest email joke, while others—even in the same company or the same cliques—just don’t
seem to get touched as often.
When you create an idea and lay the groundwork for it to become a virus, it pays to study
the vector you’d like it to follow. Why? Because there’s plenty you can do to influence its
vector, and the vector you choose will have a lot to do with who “gets” the virus. The vector
controls the hives through which the idea flows.
If you’re on the Net, for example, the barriers you erect will influence your vector. If your
site needs Shockwave and Flash and a high-bandwidth connection, you’re not likely to vector
straight into the heart of the AOL user universe, regardless of where you start. If your goal is
Unleashing the Ideavirus 94 http://www.ideavirus.com
to create a trading card mania among third graders, launching a series of cards available only
at liquor stores isn’t going to enhance the vector, even if you seed the virus by handing the
cards out at the local elementary school.
But this is about more than simple access. Remember, the goal is to market to people and
then get out of the way. So an email joke (which almost anyone with a job in this country
could access at home, at work or at the library) will still find its vector. How? There are three
factors:
1.
Who it starts with. Often, the way we decide which direction to send an idea is based on
where it came from. It’s hard, for example, to bring home a joke from the office. Instead,
we’re more likely to send it straight back into the quadrant of life from which it came.
2.
Who it resonates with . An idea has to have impact to be worth sharing at all, and we’re
much more likely to share that idea with someone whom we believe it will impact as
well. After all, if we spread ideas that don’t go viral, it hurts our reputation as powerful
sneezers. This encompasses the idea of access… I’m not likely to spread an idea if the
recipient doesn’t have the energy or the technology or the resources to get engaged with
it.
3.
What’s easy. The medium drives the spread of ideas more than you might imagine. If I
have to print something out, put it in an envelope and mail it to someone, that virus is
going to stop right there. That’s why TV and the Internet have proven to be such
powerful media for the spread of viruses—they’re easy.
Unleashing the Ideavirus 95 http://www.ideavirus.com
Medium
Scientists wasted hundreds of years looking for the medium by which light traveled. They
knew it was making it through the vacuum of space, through water and through air, but
without a medium, they couldn’t figure out how it worked.
The medium is probably the most overlooked part of ideavirus planning and construction.
It’s so obvious, we often don’t see it.
In Japan, teenage schoolgirls started and built a craze to billion-dollar proportions. They
continue to line up to use a special kind of photo booth. Here’s how it works: You enter the
photo booth (similar to the old Polaroid ones of our youth), insert a some coins and it takes
your picture.
But, instead of giving you four shots on a strip, it prints out 16 little tiny one-square-inch
images on stickers.
Now, what are you going to do with 16 pictures of yourself on stickers? Obvious—share
them with your friends! As a result, every popular Japanese schoolgirl has an autograph book
loaded with dozens or hundreds of these stickers. Sort of like your high school yearbook
signing ceremony, but on steroids.
A friend of mine, Sam Attenberg, developed and patented this technology in the States. And
while it never became a full-fledged virus in the U.S., it did develop pockets of intense
activity in certain hives. Some machines were turning $70 an hour in sticker business, every
hour on the hour for weeks at a time. In Japan, two companies dominate a multi-billiondollar
industry in Sticker Stations.
So what’s the medium? It’s the person-to-person exchange of stickers. The medium is the key
to the entire virus. Once the first person got the sheet of stickers, the only way she could use
them was by sharing them with 15 friends. But in sharing them, in using the medium
provided, she had to explain where she got them. Boom. Virus spreads.
Unleashing the Ideavirus 96 http://www.ideavirus.com
PayPal.com is another example of an extremely virulent idea that spread because the medium
was so powerful. PayPal.com is an online service that allows customers of eBay—and other
auction site—customers to transfer money online safely and securely. Now, when you pay
for something you buy on eBay, you can just PayPal.com your money to the person.
Here, the medium is the money. People care a lot about money, and since, in this case, it
solves a time-consuming problem (sending checks and waiting for them to clear), it’s
particularly welcome. And, just as we saw in the Sticker Station example, the act of using the
medium causes us to teach others about the idea.
In both cases, a focus on the medium led to the ultimate success of the virus.
Unleashing the Ideavirus 97 http://www.ideavirus.com
SMOOTHNESS: It Would All Be Easy If We Had Gorgons
The goal, of course, is to have an ideavirus so smooth that once someone is exposed to it,
they are instantly hooked. A virus so powerful that all it takes is one guitar lick on the radio,
one phrase in a book review, one glimpse of a website and you completely and totally “get
it.” And not only do you get it, but you want it. Now and forever.
One of the talents of the great Steve Jobs is that he knows how to design Medusa-like
products. While every Macintosh model has had flaws (some more than others), most of
them have had a sexiness and a design sensibility that has turned many consumers into
instant converts. Macintosh owners upgrade far more often than most computer users for
precisely this reason. We have to own that new flat panel display. We must have the new
color of iBook.
Vindigo is Medusa-like in the way the virus spreads so smoothly. It only takes one look at a
friend’s Palm in order to get hooked (and one file beaming to get it forever). The Nextel
phone has that power, and so (for some people) does Britney Spears.
Alas, it’s not going to happen for you. While you can aspire to make your product more
Medusa-like, it’s a mistake to spend all your time wishing for it to happen. The odds are long
indeed, especially if your product is not groundbreaking. The longer it takes someone to get
Unleashing the Ideavirus 98 http://www.ideavirus.com
the basic concept behind your idea, the less Medusa-like it is. But often, that’s a good thing.
Real change, and the profit that goes with it, often comes from unsettling ideas that
significantly alter the way people interact with each other and with your company. And those
ideas aren’t as smooth as some others.
Unleashing the Ideavirus 99 http://www.ideavirus.com
Persistence
In our quest for the quick hit, the easy way to start a business or just to increase our power as
sneezers, there’s a real desire for a shallow virus. A joke. A gimmick. A neat new technology
geegaw that won’t be around tomorrow.
Laser pointers are a fine example. I was in a meeting last month where the presenter used a
laser pointer to highlight various things on his deathly boring Powerpoint slides.
Unfortunately for me, not only was the presentation boring, but he kept aiming the laser at
the TV monitor, which reflected this highly focused electromagnetic radiation right at my
face, hitting me in the eye a few times. I finally got him to turn the thing off, but not
without considerable struggle.
Other than this unfortunate incident, I can’t remember how many years ago it was that I saw
someone actually using one of these pointers.
What happened was that the pointer came out, and for a few early adopters, it felt
marvelous. It touched a Jungian need in us (especially men, I think) to have a magic stick
that could project our thoughts on the wall. Of course, the best place to use one was in a
meeting of other nerds. And all the other nerds noticed the laser pointer and a virus was
spread.
But after we all went out and bought laser pointers, we discovered that they weren’t
particularly useful. After all, how much information could one really have to present that we
needed a high-tech device to point out the good stuff from the bad?
So the lasers ended up in a drawer.
In other words, the virus wasn’t persistent. Those who resisted the initial temptation to rush
out and buy a laser pointer stopped being exposed to them, and the virus died off.
Unleashing the Ideavirus 100 http://www.ideavirus.com
Compare this to the Palm virus. Every day, somebody else marches into your office,
declaring their undying love and devotion to his new pocket wonder. And unlike laser
pointers, people who love them keep using them. They persist.
In Gladwell’s terms, the Palm has now tipped in certain hives. So many people are using it so
often that you’re constantly reminded that unless you get one, you’re a loser. It’s the
persistence of the Palm more than any other viral factor that has led to its success.
Unleashing the Ideavirus 101 http://www.ideavirus.com
Amplifier
Word of mouth by itself isn’t enough. As discussed earlier, unamplified word of mouth dies
off too soon to be much good to the average business. The goal of a marketer creating an
ideavirus is to create a system that allows the positive word of mouth to be amplified (and
the negative to be damped!).
This simple idea is behind the success of Planetfeedback.com. It’s impossible for me to
understand why any business invited by Planetfeedback to participate would hesitate for even
a moment before signing up.
If a consumer has a complaint or a compliment about a company, she can go to
Planetfeedback and turn it into a letter to the company. Then, with a click, she can have a
copy of the email go to the relevant congressmen, media and regulatory agencies. Another
click can send a copy of the letter to the consumer’s ten closest friends and co-workers.
Instant amplification.
Now, if your company is the target of a complaint, which course of action makes sense? You
could either proactively grab the opportunity to stamp out a negative virus, to turn the
complainer from an angry reporter of bad news into a now-satisfied witness to how much
your company cares, or you could ignore them and hope they’ll go away. Of course, they
won’t go away. They—and the people already infected—will continue to amplify the
message.
Planetfeedback is providing a great service to all parties involved. By taking previously
invisible word of mouth and aggregating it, they’re making it far easier for companies to
understand the viruses that are already being spread, and they’re giving them an opportunity
to do something about them. And yes, they are viruses—ideas that are running amok, being
passed from person to person. At the same time, Planetfeedback gives consumers far more
power, and makes it easier for them to get attention.
Unleashing the Ideavirus 102 http://www.ideavirus.com
Of course, you don’t have to sponsor Planetfeedback. Some day they may offer a different
program… or your competitors can pay to talk to your unhappy customers instead of you.
Unleashing the Ideavirus 103 http://www.ideavirus.com
SECTION 4: Case Studies and Riffs
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 104 http://www.ideavirus.com
The Vindigo Case Study
One of the best examples of a company unleashing an ideavirus is Vindigo. You can find
them at vindigo.com on the web, and you’ll need a Palm (or something compatible) to use
the software.
Vindigo is a directory of restaurants, entertainment venues and stores in major U.S. cities.
You download it to your Palm and carry it with you. Tell it where you’re standing (in the
illustration above, you’re on the corner of Amsterdam and Broadway in New York City), and
it will show you whatever sort of restaurant or fun you’re looking for. Sorted by distance
from where you are. With ratings. For free.
What a killer app! I need to tell everyone. This is why they invented handheld computers!
IT’S SO COOL!
But, while that alone is grounds for this to become an ideavirus, as described it doesn’t seem
particularly smooth. After all, after a sneezer tells you about this cool software, you’ve got to
remember the name (vindaloo? indigo?), go home, type it into your browser, download it,
synchronize it, etc. A disaster. No way it’s going to work.
Which is where the smooth part comes in. You see, right on the bottom, underneath the
buttons for eating, shopping and entertainment, is a button that says “give.”
Unleashing the Ideavirus 105 http://www.ideavirus.com
So, when a sneezer is going on and on and on about how cool this is, you just take out your
Palm, they take out their Palm, press the give button and sixty seconds later the entire product
is now on your Palm!
That’s smooth. It’s about as close to perfect smoothness as you can get.
It goes beyond smooth. It’s persistent. The next time you synchronize your Palm with your
PC, it will automatically upload all the ratings you’ve put into the computer and get you an
updated version. Instantly. Automatically.
The ideavirus has stuck.
Is it working? Well, the folks at Vindigo seeded just 100 sneezers with the original version of
the program. Then they spent virtually nothing on advertising and waited to see if the virus
would spread. It’s now the fastest-growing application on the Palm.
Note that this isn’t viral marketing in the sense that Hotmail is. You can happily use Vindigo
for months without mentioning its existence to a friend. Vindigo works really well, but it
also happens to be optimized for spreading the ideavirus.
Unleashing the Ideavirus 106 http://www.ideavirus.com
Saving The World With An Ideavirus
The Prius is a new car from Toyota. And it’s the only car that’s ever won an award from the
Sierra Club. This is the car that’s supposed to save us from ourselves, to take a whack out of
the greenhouse effect and to conserve our remaining fossil fuels.
How? By using an engine that’s a hybrid of gasoline and electricity. By getting more than 90
miles to the gallon, giving very good performance and emitting close to zero pollution. I
dearly hope it succeeds. THIS CAR IS IMPORTANT!
Unfortunately, because Toyota is a factory-based company that uses ideas (instead of being
an idea company that owns factories) they’ve built the product completely backwards. I’m
confident that someday everybody is going to be driving a car as positive for the world as the
Prius, but it won’t be because of the way this car is marketed.
Let’s start with the name. How can you tell someone about a car you’re excited about if you
don’t know how to pronounce it? Is it pry-us, or is it pree-us? I don’t want to feel stupid, so I
just won’t say the name.
Second, is there a smooth way for me to spread the word? A visit to the Toyota website
doesn’t even show the Prius on the home page, and when I search for it, I get a very nice
page. But where’s the “tell a friend” button? How can I set up a test drive? Is there a place for
me to give my email address so I can give permission to get information on when the car is
going to be available in my neighborhood? Alas, no on all three counts.
What about a community activism component with teenagers going door to door with
petitions, hoping to lobby the local government to buy Prius police cars? Or letter-writing
campaigns that spring up from grassroots environmental organizations around the
country…?
Unleashing the Ideavirus 107 http://www.ideavirus.com
But the biggest mistake Toyota made was the way they designed the car. Unlike the VW
Beetle and the Mazda Miata, the Prius is not a driving billboard for itself. Here’s what it
looks like:
You could have 1,000 of these cars drive by and you’d never, ever notice it. You wouldn’t
notice the styling, you wouldn’t notice the gas mileage or the lack of emissions—and you
certainly wouldn’t aspire to own one just by looking at it.
Is Toyota on a mission from God? Are they acting like zealots, aggressively pushing a car that
will change the world for the better, the most powerful idea to come out of the car industry
since Henry Ford perfected the assembly line? We need passion from our manufacturers.
What a lost opportunity! An idea merchant in search of a virus would take a very different
tack. Instead of trying to make it cheap and boring, they’d realize that the first people to buy
a car like this are people with money to risk on an unproven technology. Realize that the
opinion leaders and nerds who are most susceptible to this idea are also the most likely to
want to drive an exceptional car.
I’d redesign the thing to be stunning. Different. Unique. Maybe a permanent bumper sticker
announcing my current gas mileage on an LCD readout. Or a fleet of far-out colors. The
first 50,000 people who buy this car will be doing it to make a statement. And every person
who does will be making that statement to the 1,000 or 10,000 people who see them driving
it. A virus waiting to happen.
Remember what I said about the VW Beetle? 180° difference.
Unleashing the Ideavirus 108 http://www.ideavirus.com
Toyota forgot to pick a vector for this car. They don’t know exactly who they want to buy it,
so they designed it for everyone. Precisely the opposite strategy of the new VW Bug. But
remember, an ideavirus adores a vacuum, and there is a very big and very empty vacuum just
sitting here, waiting to be plucked. Toyota could have picked any vector they wanted,
leading to any hive they chose, and yet they chose none.
And finally, I wouldn’t let just anyone buy the first models off the line. I’d select the very
best sneezers, the loudmouths, the pillars in their community and do whatever it took to get
these folks to drive a car. James Bond? Julia Roberts in her next film? The mayor of Carmel,
California or the head of Greenpeace?
This is urgent. This isn’t about making another few million bucks from a website. It’s about
infecting the population with a good virus, and doing it before the vacuum fills up with junk
and it’s too noisy to communicate about it.
Unleashing the Ideavirus 109 http://www.ideavirus.com
Is UNLEASHING THE IDEAVIRUS An Ideavirus?
Here is the step by step plan I’m using to turn this manifesto into an ideavirus:
1.
Describe something important and cool and neat and useful and new, and do it in
compelling, clear and exciting words.
2.
Launch the virus to the largest audience of sneezers I can find. In this case, that means
the readership of Fast Company. Do it with graphic splash and panache and impact.
3.
Make it smooth. Post the entire manifesto at http://www.ideavirus.com. Include commentary
from big-name authors, pundits and industry professionals. Include the entire text of not
just the manifesto but the entire book. Make it easy to send the book to a friend. Include
an audio version of the book. Include my powerpoint slides. All for free.
4.
Run ads to create an environment in which sneezers feel comfortable spreading the
manifesto to others.
5.
Maintain the virus as it grows by doing speaking engagements and distributing free
copies of the hard-copy version of the manifesto to appropriate sneezers.
Unleashing the Ideavirus 110 http://www.ideavirus.com
Moving Private To Public
One of the challenges facing oldline companies as the ideavirus becomes more important is
that they’re used to providing private services. Your friends and acquaintances probably have
no idea what brand of PC you have, whether you have gas or oil heat, how often you see the
chiropractor or what your favorite kind of wine is.
Because of the private nature of these relationships, the only way to expand the market for
them is for the marketer to spend more money and interrupt more people with more junk
ads. BUT, if they can figure out how to make them public, if they can figure out how to
launch an ideavirus, the whole equation changes.
Here’s an example: your frequent flyer miles.
American Airlines has made a fortune using frequent flyer miles to induce loyalty, and just as
important, to establish a currency that they sell to other companies.
But none of your friends really knows your frequent flyer habits. You almost never talk about
them unless something exceptional happens that you want to brag about… like buying
tickets for the whole family to fly to France with your miles.
There are a number of things that American Airlines can do to move miles out of the closet
and turn them into an ideavirus. For instance, they could allow people to buy, sell or trade
their miles. Would this lead to more mileage redemptions (a bad thing)? Sure. But it would
also turn miles back into a nationwide fascination.
Far more clever would be to make the following announcement at a convention jammed
with business travelers: “If you can find someone at this convention who has precisely the
same number of miles as you do, we’ll give you both a million miles.” Suddenly, every
person you meet wants to talk to you about your mileage status.
Unleashing the Ideavirus 111 http://www.ideavirus.com
Hakuhodo, one of the largest ad agencies in Japan, used a similar approach and turned it
into a national craze. It seems that sending New Year’s cards is a big deal in Japan…much
bigger than Christmas cards.
Most people buy their cards at the post office—envelope and stamp included. When you
send a card, it comes with a lottery ticket, good for a small prize if the recipient wins (a
bicycle, a radio, etc.).
Hakuhodo runs their promotion on the Net. And the cards are free to send (no stamps, no
fee). But the best part is that if the person you’re writing to wins, you win the same prize. So,
the more you send, the happier your friends are, and of course, the happier you are.
Not only did this promotion go viral, it turned into an epidemic. In 1998, 25% of the
people with Net access in Japan either sent or received one of Hakuhodo’s cards. And
Hakuhodo cashed out by selling ads in each and every one of those cards.
In order to turn these public ideaviruses into useful, long-term assets, the companies that
create them need to gain permission from people to follow up directly. Then, they go back to
private marketing, at a very low cost, with excellent results, until they’re ready to go public
again with another virus.
Of course, going public doesn’t mean you have to run a selfishly oriented promotion. When
I was in college, the gay and lesbian center ran a campus-wide activity called “Wear jeans on
Wednesday if you’re gay.” Suddenly, something that had been a private topic was now the
topic of discussion among everyone. If you weren’t wearing jeans, was that because you were
afraid that people thought you were gay? Is there something wrong with being seen as gay,
whether you were or not?
One simple act turned the notion of sexual preference into an ideavirus and generated
thousands of hours of intense discussions about how society (and how we) viewed the issue.
Unleashing the Ideavirus 112 http://www.ideavirus.com
YouÕre In The Fashion Business!
Without question, the most difficult part of unleashing a manifesto is creating something
that’s virusworthy. And one of the key components of that art is understanding the fashion
moment.
Why do open-toed shoes come and go? Bell-bottoms? Miniskirts?
How is it that every year, multiple clothing designers launch very similar clothes, without
consulting with each other in advance?
Source: Corbis
Why is it that we rarely see people dressed like the two women above? Did these folks wake
up one morning and go out and buy the entire outfit at once, or did it happen gradually?
Why do some Internet businesses (group scheduling, free email, health portals) seem to
appear simultaneously, even though it took them months or years to launch?
Unleashing the Ideavirus 113 http://www.ideavirus.com
The fashion moment occurs when a respected hive member takes a chance and tries out
something new.
One of two things occurs when a hive member shows up with a new “outfit”:
1.
The hive embraces the new. They start wearing a nose ring or get a tatoo or switch from
using a Filofax to using a Palm. When this occurs, the respected member gets MORE
respect, becomes more influential and reinforces his position as a powerful sneezer.
2.
The hive rejects the new. Many times, the person who introduces the new item will be
ignored or ridiculed (this happens more to the less-respected members of the hive, but it
happens to everyone sooner or later). When this happens, the person who tried to
introduce the new fashion loses respect, becomes less influential and is usually less likely
to try again in the near future.
Obviously, respected members are hesitant to lose their positions of influence, hence the
consistency and uniformity among hives.
Some hives are incredibly conservative (go to the Assocation of American Actuaries annual
meeting and you won’t see an awful lot of surprising new innovation), while others are
known for their daring (the trends demonstrated on the New York City nightclub scene
oscillate like the NASDAQ).
Some people—I call them fashion editors—seem to have an innate sense for knowing when a
hive is ready to adopt a new virus. Successful venture capitalists, journalists, chefs, research
and development labs and record label executives are great fashion editors.
Clive Davis at Arista Records was a stellar fashion editor in the music business for
generations. He launched dozens of breakthrough acts… from Aretha Franklin to Whitney
Houston, Carlos Santana to Patti Smith. The only thing they had in common was that they
were just right for their time. A month earlier or a month later and they might never have
succeeded. (Well, maybe Aretha would have succeeded no matter what…)
But no fashion editor is infallible, and if they’re not careful, they fall into one of two traps:
Unleashing the Ideavirus 114 http://www.ideavirus.com
They lose touch with the hive and fall in love with their own taste. Without the feedback loop
the hive provides, they “lose their touch.” Someone who had a seemingly hot hand starts
failing.
Warren Buffet is a brilliant stock market investor with an extraordinary ability to understand
what other people are going to want to invest in. But when Internet mania started to hit the
stock market, Buffet lost his ability to predict what the hive would predict. I think he DID
know, but overruled his sense of what would happen with his own common sense. Buffet left
billions of dollars of profit on the table because he refused to believe that the Internet stock
ideavirus would spread across the hive of investors.
They stop thinking of themselves as fashion editors and start to believe that they are fashion
makers. Rather than acting like someone who has a sense as to what virus will hit the hive
next, they believe that they are respected enough by the hive to FORCE them to accept the
next virus.
Fashion designers are famous for this, as are rock groups, authors and product marketers.
Take a look at New Coke—the biggest flaw in the introduction of this product was that
Coke believed that if they willed the consumer to adopt a new formula, the consumer would
do as they were told. Instead of spreading like a virus from a respected hive member, they
tried to ram the formula for New Coke on the hive. The hive rejected it.
The challenge your business faces is finding or training a fashion editor. Launching products
too early is just as bad as launching them too late—if you miss the timing, you fail to fill the
vacuum with your virus. Miss the timing and the profit belongs to someone with better
timing and better fashion sense than you.
To those dedicated to the idea that your business is a factory, all this must sound like heresy.
After all, if you wanted to go into the fashion business, you’d have gone into the fashion
business! But, like it or not, we’re all in the fashion business.
Unleashing the Ideavirus 115 http://www.ideavirus.com
A few years ago, there was plenty of cherry wood to go around. People weren’t making much
furniture out of it… it wasn’t in style. Then a furniture designer named Thomas Moser
decided that his fashion sense was telling him that cherry wood would make a comeback.
That once people saw how beautiful the wood was, the idea of furnishing your house in this
warm, comfortable wood would spread through his chosen hive.
Moser built an entire company around cherry wood furniture, and bought thousands of acres
of prime cherry in anticipation of demand. Today, Thomas Moser has grown more than
30% a year for the last ten years, with showrooms in New York and overseas selling $5,000
tables and $3,000 chairs. Not because the furniture is great (which it is) but because he
created a fashion that resonated with his hive, because he launched an ideavirus.
Unleashing the Ideavirus 116 http://www.ideavirus.com
The Money Paradox
The sooner you ask for money, the less you’ll make.
The single biggest mistake idea merchants make is that they ask for money too soon. On one
hand, you want to charge early and often, so you don’t waste time on people who are just
looking, and so you can maximize your income before your idea fades. “Take the money and
run” is a cliché for a reason.
But this strategy introduces friction into the system. Many marketers require people to pay
the most when they know the least. For example, why don’t movie studios run a day of free
sneak previews to get the virus started, and then charge more once everyone wants to see the
movie? Today, if you want to taste a new movie, you’ve got to pay $8 for the privilege.
On the Internet, dozens of new businesses have discovered how important this model is. A
company called eFax offers a service that lets you get faxes delivered to your email box. They
launched it as a totally free service. Why? Because it’s scary enough to be one of the first
people to try something as flaky as eliminating your fax machine. And it’s even scarier to pay
money for the privilege as well…
So eFax has a plan: get people hooked on a free system. Build an ideavirus. Then upgrade
people to a paid system that offers all sorts of extras.
1. Fill the vacuum
2. Achieve lock-in
3. Extract revenue
They can fill the vacuum by getting in first and furious and spreading the virus. They can
achieve lock-in by making it hard for people to switch to a competitor (what a hassle to keep
changing your fax number!). And finally, they can extract revenue by offering value-added
services or selling advertising.
In that order!
Unleashing the Ideavirus 117 http://www.ideavirus.com
Will eFax be guaranteed an easy upgrade path to paying customers? I have no idea. Some
businesses (like email) will be stuck at FREE forever, thus making the whole journey hard to
justify. In this case, they could offer free faxes with an eight-hour delay before you get them,
but for $5 a month, you get the faxes instantly. So it’s free for me to try out, free to spread,
but profitable after lock-in is achieved.
The challenge, of course, is to figure out which businesses have a payoff at the end. The
challenge is also to be patient enough to wait, to introduce the friction of charging at just the
right moment.
Watts Wacker catapulted his career by writing The 500 Year Delta. After the book came out,
people started to hand it around, to embrace his ideas. This led to larger audiences and a
dramatic increase in bookings for speaking engagements. In a few months, I’m confident he
made more in speaking fees than he had from royalties on the book. By letting the ideavirus
grow before trying to extract much profit, he was able to make more money in the end.
In very transparent markets like the Internet, the fear is that all ideaviruses will be so
competitive that you’ll never be able to extract money. That’s why the race to fill the vacuum
is so intense. If you can fill the vacuum aggressively and permanently, it is far easier to extract
money.
Unleashing the Ideavirus 118 http://www.ideavirus.com
Think Like A Music Executive (Sometimes)
There are plenty of lessons you can learn about viruses from folks in the music industry
(current behavior notwithstanding, but more on that later).
First, industry executives realize that nobody buys a CD because they like the quality of the
polycarbonate disc. If you don’t like the idea of the music, you’re not going to buy it.
Second, they realize that making money later is way more important than making money
now. They learned this the hard way. Consider radio for a second. Before radio, music sales
were tiny. Why would you buy a song for your Victrola if you’d never heard it before? How
could you know if it was any good?
At first, radio might seem like a threat to the recorded music industry. After all, they play the
ENTIRE song, not just a few notes. And if it’s a hit song, you can hear it night and day on
the radio every few minutes if you’re so inclined.
For a while, the music business fought the idea of radio stations playing songs for little or no
compensation. Then, in the 1950s, they realized how valuable airplay was—so valuable that
a congressional inquiry discovered that music labels were bribing disk jockeys to play their
records.
Fast forward a few decades to MTV. Once again, the music labels balked at supporting
MTV’s insistence that they provide expensively produced music videos—for free! It took a
year or two for them to discover that MTV made hits—that giving away the music for free
turned out to be the best way to sell the music.
Music execs know that you’ll pay nothing to hear a song on the radio, but if you like it,
you’ll gladly pay $15 for the CD. And that if you love the CD, you’re more likely to pay $40
for tickets to the local concert, where you might be converted to a raving sneezer, much more
likely to infect your friends and neighbors with raves about the band, the song, even the
souvenirs!
Unleashing the Ideavirus 119 http://www.ideavirus.com
For some reason, history is repeating itself. Rather than embracing Napster, the software that
lets millions of people listen to each other’s CD collections, music moguls, fronted by the
hard rock band Metallica, are once again complaining about the free distribution model.
Even if the record companies are able to beat Napster in court (a likely outcome) it won’t
matter. There are already dozens of technologies (like gnutella) waiting to take its place, and
each will be harder to stamp out than the one before.
Patience! Instead of hassling Napster, they ought to figure out how to license Napster and
the others, probably in exchange for intensive promotion of their hottest acts. Why not let
me subscribe to my favorite bands, paying for live performances or attending private concerts
or buying T-shirts. I’m certain that if the Grateful Dead were still around, their primary
income source would be souvenirs, followed closely by live concerts. Is that what the record
companies want? Doesn’t matter. It’s what the network is going to deliver, regardless of how
they feel.
Is the CD going to disappear? Absolutely, regardless of what happens to Napster. What will
determine the future of the record business is whether music execs are able to redefine their
jobs around what happens after they ignite a virus over Napster or its successor.
Unleashing the Ideavirus 120 http://www.ideavirus.com
Is That Your Final Answer?
When a sneezer is ready to spread your ideavirus, what should he say?
It sounds like a simple, almost silly question, but it goes to the core of how smooth you can
make your virus. If you give sneezers easy-to-follow, effective instructions, they’re likely to
follow them, because, after all, their goal is to spread the virus.
On “Who Wants to Be a Millionaire?” the producers insist that Regis Philbin repeat the
catchphrase, “Is that your final answer?” almost to distraction. But now it’s become a
powerful, smooth tool for sneezers who want to spread the virus. I must have heard the
phrase fifty times and read it in dozens of newspaper columns before I saw the show for the
first time.
By giving loyal watchers a five-word catchphrase, the producers created (intentionally or not)
a powerful shorthand for referencing the show. Hotmail did the same thing with the sig file
in the free email each person sent. Right there at the bottom of each email, with no
additional work on the part of the sneezer, were specific instructions on how to get Hotmail.
Buffalo Springfield and the Beatles did the same thing with some of their songs. It took just a
few notes—an investment by the listener of seconds, not minutes—for them to expose their
“idea” to a new listener. By working so hard on the first chords of the song, pop music
producers (and Beethoven for that matter) made their products far smoother. It’s easier to
share the song when you can hum the riff.
For most ideas, the web can be a powerful tool to help with this. What might a website for
sneezers look like?
The first touch, the first impression and first visit, must go beautifully. It’s got to be fast.
It should contain exactly what you’ve tested and discovered that most effectively captures the
attention of the first time-visitor. You’re in control in this moment, and you can make it
work or not.
Unleashing the Ideavirus 121 http://www.ideavirus.com
The site should also be filled with tools that make it really easy for a visitor to become a
sneezer. Get out of the way. Give the sneezer something to share. Do like Tom Peters (at
http://www.tompeters.com) and include all your Powerpoint slides. Don’t require registration or
permission at this stage. Let them in, sell them on the idea, then give them the (free) tools to
share.
Unleashing the Ideavirus 122 http://www.ideavirus.com
A Dozen ideaviruses Worth Thinking About
Company Big idea How you spread the virus (the
medium)
Polaroid Instant photography “HEY! Look at this,” you say at
the party.
Tupperware The best food storage devices Get your friends to sell their
friends—multi-level marketing
Fax machine Documents delivered by phone The more you sell to your
business associates, the better
your machine works.
Home Shopping Network Shopping via cable TV “Hey Madge! Look what’s on
Home Shopping,” you say to
your friends on the phone.
Fast Company Journal of the new economy Company of Friends—monthly
meetings of local fans of the
magazine.
CarmineÕs Restaurant Tons of food, tons of garlic Six-person minimum for
reservations—you need to sell
your friends to get in.
Beany Babies Collectible teddy bears If other people start collecting,
your collection increases in
value.
Gamesville Super sticky games on the web Word of mouse—email your
friends and invite them over.
Hotmail Free email Totally viral…every mail you
send promotes it.
Tommy Hilfiger Urban preppy chic Logo virus—the more you wear
the logo, the more people see it.
ÒThe Cathedral and the
BazaarÓ
Open source programming
works
Enabling powerful web sneezers
to spread the word by giving
them a powerful manifesto they
can share.
Vindigo Zagats on my Palm “Give it to me,” and a friend
can beam it over in seconds.
Unleashing the Ideavirus 123 http://www.ideavirus.com
Why I Love Bestseller Lists
One of the critical moments in the spread of an ideavirus is the question the consumer asks
before diving in: “Is it worth my time/money?”
Of course, your recommendation is important to me. Of course, I want to look as good as
you, be as smart as you, have as much fun as you. But I also care desperately about everyone
else’s opinion. After all, none of us is as smart as all of us!
The most common way this popularity is reinforced is that the user will hear about a new
ideavirus from more than one person. Usually, we hear about something first from a
promiscuous sneezer, someone who has some sort of benefit from making the
recommendation, or at the least, someone who’s always recommending stuff. We all know
somebody who eats out every night or listens to every CD or is into whatever bizarre
conspiracy theory has gripped insomniacs this week.
But then, sometimes we hear about the same ideavirus from someone else. And then another
person. Finally, we realize that something is really going on, and we investigate.
In the real world, these reinforcements are usually caused by sightings or physical
interactions. Riding through the New York subway last year, I encountered a kid wearing
what appeared to be a black stocking on his head. But along the hem were the words,
“Tommy Hilfilger.” It seemed like an odd affectation and I let it go.
A week later, I saw four more Hilfiger skull caps. In the week after that, a dozen. If I were in
search of genuine urban chic, I certainly would have bought one at that point, if only to
protect my trademark bald pate from the winter chill.
The same thing happened with the VW Beetle. First there was one in my neighborhood (a
yellow one) and then a few, and then a dozen. With all these reinforcements, I assumed that
it was now a safe thing to consider, and went to the dealer to have a look for myself.
Unleashing the Ideavirus 124 http://www.ideavirus.com
Online, the rules are very different. There is no physical world to bump into. Instead (and
even better for the statistician in each of us), there are actual digital counters and accurate, up
to the minute bestseller lists. No guessing. No inferences. The real scoop.
Amazon.com has a bestseller list more than a million titles long. Visit any title and you can
see where it stands compared to every single other title in the world. Wow. Now we instantly
understand what’s hot and what’s not.
MP3.com has done the same thing with music. As a track gets played more and more often,
it moves up their digital bestseller list. And yes, Zipf’s law works here too—the topmost
tunes are downloaded far often more than those just below them.
We use the same math when we look at the MediaMetrix list of the most visited websites, or
Variety’s tally of the weekly box office numbers (some people saw “Titanic” just because it
seemed that everyone else was). Various organizations also track bestselling cars, bestselling
vodka and highest-paid executives.
One of the best ways to facilitate adoption of your ideavirus is to find a bestseller list that
makes sense and then dominate it. If that’s impossible, figure out how to create your own
bestseller list and popularize that!
This isn’t just conjecture. A breakthrough paper by Stanford Business School professor Kirk
Hanson demonstrated this in a really profound way. His team artificially boosted the
bestseller status of files for download on the web (they downloaded one file over and over
again, increasing the counter of how often it had been downloaded). The result? Heavily
downloaded files get downloaded more often! Nothing was changed but the counter, but
users were more interested in seeing the most popular files. Simple, but true.
Want to launch a new drink using your company’s chi-chi liquer? Why not identify the right
bar, frequented by powerful sneezers in the hive you’re targeting. Then pay the bar to post a
“bestselling drinks list.” Now, bribe enough folks to go in and buy themselves a drink. Soon,
you’ll see your drink climbing the bestselling drinks list, and this alone ought to be enough
to get other—less easily bribed drinkers—to give it a try.
Unleashing the Ideavirus 125 http://www.ideavirus.com
Of course, sampling doesn’t always lead to the spreading of a virus, but without sampling,
you’ve got no chance, do you?
Unleashing the Ideavirus 126 http://www.ideavirus.com
How A Parody Of Star Wars Outsold Star Wars
According to USA Today, a parody called George Lucas In Love is currently outselling the
new Star Wars movie on video on Amazon. How is this possible? How can mighty
Twentieth Century Fox be beat by a nine-minute, $8 handmade film?
Because the parody is an ideavirus. And because the medium of the Net is the perfect place
for the word to spread.
In the old days, if you made a movie, you needed movie theaters across the country to show
it. That’s way outside the reach of an entrepreneur, regardless of how clever his movie is.
Videotape leveled the playing field a bit (Blockbuster can carry hundreds or thousands of
titles) but it’s still very difficult, time-consuming and expensive to force your way into
nationwide distribution.
But Amazon is a different story. Amazon prides itself on carrying just about everything. Since
they don’t have to carry much inventory, Amazon doesn’t take much of a risk by listing a
title. And the entrepreneur can certainly find his tape listed along with the thousands of
others available.
Unleashing the Ideavirus 127 http://www.ideavirus.com
So distribution is the easy part. But how to spread the idea?
Well, the parody fills a vacuum. In this case, the vacuum was “funny and interesting news
about Star Wars.” Certainly, the launch of the videotape was a yawner, the mania about the
film version having largely subsided. Would many people buy the video for their libraries?
No doubt. But it wasn’t news.
But now, here’s an email telling me that someone has seen the funniest little video. It’s
hysterical, my friend says. So I click on over to Amazon (using his affiliate link, I notice—he
may be a powerful sneezer, but he’s also making a profit on this virus). There, I note more
than 100 reviews, all of them positive. I see that it’s a bestseller. I realize that there’s almost
no risk here, certainly worth ten bucks and a few minutes of my time. I buy it.
And after I see it, I’ll tell five friends. This time using my affiliate relationship.
A classic ideavirus. Yes, it would have grown faster if the filmmaker had just put the video
online for free, but he was stuck in the mindset of making money now. Yes, the charge and
the wait for shipping definitely slowed the virus down, but at the same time, it was a nice
balancing act—a slightly slower virus in exchange for tens of thousands of dollars (and
probably a contract for a real movie from a studio).
If it were me, I probably would have posted a low-resolution excerpt of some of the funny
parts online… it’s going to happen anyway, so the filmmaker might as well do it and thus
control what the sneezers say while also increasing the velocity of the virus.
Unleashing the Ideavirus 128 http://www.ideavirus.com
Wassup?
I first heard about the Superfriends parody in an email. Apparently, some clever animator
had taken the soundtrack of the ubiquitous Budweiser commercial and replaced the video
portion with Batman, Superman and Aquaman hamming it up and having a few brews.
Clicking on a link (pretty smooth transition from interest to exposure, you’ll notice), I see
that it’s on a reputable site and happens to be one of the most downloaded files (a bestseller
list!).
Soon, I’m laughing out loud. It really is funny. Of course, I’ve got to tell three friends, so I
do. It’s going viral.
A few weeks later, a site launches another Wassup parody. This one uses the AP photo of
Elian Gonzales as the star. But this time, the virus grows far faster, with more than 100,000
people seeing it in less than 24 hours.
Why did it grow so fast? Because everyone who had seen Superfriends and liked it didn’t
need much coaxing to get infected by this one. By tapping into a virus-friendly base, it took
much less effort for the marketer to get the message to spread.
(Of course, this is a paradox, because the ideavirus loves a vacuum. In this case there wasn’t
a vacuum—the Wassup parody was an old joke. So, in order to make an impact, it had to be
fresh and at least as funny. But once it cleared that hurdle, the rest was taken for granted.)
This time, though, the ending of the cycle was very different. The Associated Press fired off a
letter to the site behind the virus, claiming copyright infringement and not interested at all in
the idea of parody and its protection. So the short movie came down.
Inevitably, if you create a piece of digital media that becomes popular, someone is going to
parody it, or at the very least, use it in a way that you’re not delighted with. If your digital
media becomes that popular, odds are you should embrace it, not fight it. Budweiser, for
Unleashing the Ideavirus 129 http://www.ideavirus.com
example, has wisely let the parody virus spread unfettered. Being parodied online is a
shortcut to burning the Budweiser brand further into our subconscious.
Unleashing the Ideavirus 130 http://www.ideavirus.com
Judging a book by its cover
No question, a great cover can make or break your book. Kurt Andersen wrote one of the
funniest books I’ve ever read (Turn of the Century) but, by all accounts, it didn’t meet sales
expectations. Why? One reason is the cover, which is one of the worst I’ve ever seen in my
life.
Remember, the search for Medusa is usually a hopeless quest. But just as it’s difficult to sell
someone on your ideavirus with just an image, it’s also nearly impossible to suck them
further in if the image is offputting, inconsistent or boring.
Boring is probably the worst offense. Whether your product is a book, a trading card, a car
or even the tag on a bag of tea, boring is the obvious, but wrong, solution.
You’ve worked very hard on the stuff “inside.” You’ve refined, tested, edited and slaved to
make sure that the idea is powerful indeed. And then it comes time to make the
package—the cover. The prevailing wisdom is to create a cover that’s attractive but not
offensive. Something that will attract attention from everyone and offend no one.
This is nonsense, of course. It can’t possibly attract everyone and offend no one. The very
best cover images are like a cold glass of water thrown in your face. They break one or more
rules of graphic design or industry rules of thumb. They play off existing images but change
them in a vital and important way. They’re loud. They attract the eye, but they also hold it.
And most of all, they intrigue us enough that we need to understand what’s inside: we set
ourselves up to be exposed to the virus.
When Yahoo! first launched, the company name and logo broke every rule in the book. But
co-founder Jerry Yang will be the first to tell you that in a world populated with Lycos,
AltaVista, InfoSeek and Architext/Excite, Yahoo! was the easy winner. Easy to spell. Easy to
type. Easy to tell other people about.
And it had personality. It meant something.
Unleashing the Ideavirus 131 http://www.ideavirus.com
Was it risky? I don’t think so. A boring, hard to spell, meaningless name like Lycos was risky.
Unleashing the Ideavirus 132 http://www.ideavirus.com
Being The Most
Turns out there’s been a battle going on for a few years—the battle to make the hottest hot
sauce in the world.
At the beginning, you made a hot sauce by using peppers. Hotter peppers made hotter sauce.
And a sauce made from Scotch Bonnet Peppers (the hottest peppers on Earth) was the
hottest sauce on Earth.
Then, some nutty scientist figured out how to extract just the essence of hotness from Scotch
Bonnet Pepper puree. By using gas chromotagraphy or some other evil technology, he was
able to create a sauce more than 1,000,000 times spicier than your basic pepper.
Why does this matter? Because being the hottest hot sauce ever made is like being the Mona
Lisa. Because if I’ve managed to eat chili with the hottest hot sauce ever made in it, I’m
going to tell my friends. I’m going to spread your hot-sauce virus. If it’s the second or third
hottest, who cares?
There’s always room in any list for the world record holder. The greatest basketball player
who ever lived, or the nastiest restaurant owner, or the fastest computer. It’s noteworthy. It’s
news. It’s worth sharing.
My dad’s hospital crib company dominates the market, making most of the cribs that are
used in hospitals around the world. Their standard models cost $700 to $2,000 each, and
they last forever. How to grow the business? How to get more attention and more sales?
His engineers found a leading hospital, and together they designed the best (and the most
expensive) hospital crib in the world. With all the options, it costs about $7,000. Yet it’s
selling like crazy, from the Philippines to Tuscon. Why? Because it’s worth talking about.
Because it embodies an idea, and it’s an idea worth sharing.
Google.com has plenty of traffic, yet they’ve never spent a nickel on advertising. How?
Because it’s the fastest and most complete search engine ever built. Electronically amplified
Unleashing the Ideavirus 133 http://www.ideavirus.com
voices—from nerds to magazines—are happy to trumpet the idea that there is a faster, better
way to search than using the tried and true favorites.
As you think about a corporate virus or a personal one, consider: What you are the best or
the most at? How can you refine and amplify those traits to create a Wow! product…a
world’s record holder that is worth mentioning?
And by the way, if you’re not facing a vacuum (and most of us aren’t lucky enough to be in
that position) you’ve got to be ten times better than what’s already there, if you’re going to
start your own virus.
Unleashing the Ideavirus 134 http://www.ideavirus.com
In Defense Of World Domination
Targeting isn’t enough. Being a world record holder isn’t enough either. You also need to
dominate your hive.
Having 5,000 loyal, rabid fans of your ideavirus is great. Unless, of course, your audience is
the population of Massachusetts. To dominate Massachusetts you need a lot more fans than
that. Without the power of reinforcement, your virus will peter out. Unless individuals are
hearing from sneezers again and again, your virus will slow and will probably die out over
time.
Imagine, instead, that you have 5,000 fans at Stanford University, which is a hive with a
population of about 15,000. The chances that you’ll be exposed to every other member of
the hive is huge. Why? Because if each one of the 5,000 fans tells just a few other people,
you’re already hitting each person more than once. And if the idea is virusworthy, that’s
probably enough to dominate the entire campus. Even the laggards will surrender when they
see everyone else is doing it (even the accounting department at my old company did the
Macarana at the company Christmas party).
Malcolm Gladwell calls this the tipping pointD—the idea that creating and propagating an
ideavirus is not enough. The biggest win comes the last time your virus doubles in size. The
biggest win comes when you’ve so dominated the hive that the last folks (who are often the
most profitable folks) can’t help but come along. They tip because they hear from so many
respected sneezers that they feel they have no choice but to get on the bandwagon.
This happened with AOL. A few years ago, AOL was paying $300 in marketing costs to get
one new member. All those CDs that showed up in every magazine were expensive, but they
were effective.
But how could AOL justify spending $300 to get a member who had a lifetime value of just
$124? Jan Brandt, the genius behind the campaign, realized that if she could win at this
expensive part of the curve, the game would soon turn in her favor. She knew that once she
Unleashing the Ideavirus 135 http://www.ideavirus.com
got over the hump and dominated the hive of people about to go online, the next generation
of users would come along far cheaper.
She was right. Once AOL established dominance for new users, they established a network of
powerful sneezers. Powerful, because these were folks who until quite recently had been new
users. These once-new, once-lost users had the credibility to spread the word to those just
behind them on the learning curve. They were powerful because they’d been there, and their
personal experience counted for more than any salesperson’s could.
The virus had authority, because every “bestseller” list credited AOL with being far and away
the most popular Internet service provider in the land. Today, someone at Sun City who
until recently had no idea what they were talking about when they said “Internet” could
proudly recommend AOL to the person in the next condo. AOL now spends about $100 in
marketing to get a new member—because their virus tipped.
There’s plenty of interesting action that occurs before the tipping point, though. Viruses
need to spread before they tip, and a smart marketer can be quite happy indeed along the
way.
Dominating the hive is essential in starting the virus in the first place. And most marketers
make the mistake of picking too big a hive to focus on in the first place.
If you go to the Consumer Electronics Show in Las Vegas, you’ll see one of the largest trade
shows in the world, and you’ll also see hundreds of companies spending millions of dollars
trying to dominate the show. All of them fail. Which is why it’s so rare for a virus to be
launched at the CES. It’s just too noisy, and there are no exciting but safe recommendations
for the most powerful sneezers to make.
The smart marketers take a different path. They launch at Demo or Spotlight or Esther
Dyson’s conference—a much smaller venue, but a higher concentration of powerful sneezers.
Here, for about the same money as making a whisper at CES, you can completely dominate
the discussion.
Unleashing the Ideavirus 136 http://www.ideavirus.com
If YouÕre A Member Of The Academy, You Go To Movies For Free
If there’s an association of powerful sneezers, it’s the Academy of Motion Picture Arts and
Sciences. This association of actors, screenwriters and directors has celebrated movies every
year for nearly a century, and every year it seems to get more popular and more influential.
If your movie wins an Oscar, you can count on a blip at the box office, and even better, a
long, profitable life on video. And of course, any actor who wins one has a label that will
enhance his career forever.
So, how much do the studios charge Academy members to go to a movie? That’s right,
nothing. Not only that, but the studios are delighted to deliver the latest movies to an
Academy member’s home, on her choice of VHS or DVD.
Why? Why give the movie away? Well, here it should be pretty obvious. The leverage that
comes from building buzz among Academy members more than pays for the cost of sharing
the movie with them. In fact, the benefits are so obvious that studios like Miramax have been
accused of trying to buy the Oscars by throwing hundreds of thousands of dollars of trade
advertising at Academy members.
Well, if this is so obvious, why bring it up? Because your idea, regardless of marketplace, has
a similar group. Maybe it’s not as easy to find or as easy to reach, but there are powerful
sneezers in the audience for almost every idea. It’s the money paradox, but on a much smaller
scale. Finding these sneezers and giving them a sample of your idea for free is a no-brainer.
Even better, figure out what it costs to deliver it with impact.
I met with a marketing executive from Hong Kong last week. He’s building a company that
is targeting the health care and financial services industries. He’s got a big idea, and if he can
persuade some of the key sneezers in the industry, then most of the other companies are sure
to follow.
Unleashing the Ideavirus 137 http://www.ideavirus.com
The good news is that he was invited to speak at a gathering of 100 top chief information
officers from the financial services industry he’s targeting. The bad news is that he was
planning just to give a speech.
What an opportunity! What a chance to talk to all the key sneezers at once and dominate the
hive. We did the math, and it’s clear that even if he needs to buy each one of the attendees a
BMW to get their attention, it’s worth it.
When you have an opportunity to dominate not just a hive, but the sneezers in the hive, you
need to spare no expense to do so. Don’t just give a speech about how your product works
well. Fly in three satisfied customers to tell their stories in person. Don’t just give a speech
and ask for questions. Sponsor a cocktail party afterward so you can meet individuals and
answer their questions. Don’t just give a speech about how your product is safe and secure.
Give each attendee a first aid kit for their car. By focusing on this key moment, by overinvesting,
you can lay the foundation for a virus to come later.
Unleashing the Ideavirus 138 http://www.ideavirus.com
How An Ideavirus Can Drive The Stock Market
When you think about it, the stock market is nothing but thousands of ideaviruses. (That’s
right, thousands. An ideavirus doesn’t have to dominate our entire culture to be an
ideavirus… some last for just a few days in a very isolated hive, then disappear.) When you
buy a share of stock, you don’t really get anything—just the right to sell that stock to
someone else tomorrow. So… if a positive virus catches on and the demand for the stock
skyrockets, you win.
The market’s respect for ideavirus thinking starts before the company even goes public.
Choosing an investment bank for your IPO is a first step. Firms like Goldman Sachs and
Alex.Brown are powerful sneezers (even though they can easily be bought off with millions of
dollars in investment banking fees by eager companies looking to go public). If one of these
firms aggressively recommends the stock to institutions, the virus starts off on the right foot.
The alternative—marketing the stock through a smaller, less respected (and perhaps cheaper)
investment bank—is almost certain to lead to a lower return.
The next step is pricing the IPO. The current rage is to underprice the stock being offered to
the public, because that will lead to a huge first day appreciation in the stock. It’s not
unusual for an IPO (like Globe.com, Martha Stewart Omnimedia or Street.com) to
dramatically increase in price on the first day of trading.
Why do this? Why leave all those proceeds on the table so that the folks lucky enough to buy
into your IPO make the money instead of your company? The answer is simple, and it has
two parts:
First, by rewarding the powerful sneezers who are lucky enough to buy into your IPO, you
maximize the chance that they’ll participate and will tell their less powerful (but more
numerous friends) about this exciting new investment.
Second, the rapid rise in the first day of trading allows other powerful sneezers (the news
media and brokers you don’t have direct contact with) to talk with excitement and
amazement to the next group of potential investors. In other words, this is cheap marketing.
Unleashing the Ideavirus 139 http://www.ideavirus.com
It’s a way of communicating news (this is a hot stock) to large numbers of people in a
powerful way.
After the stock is public, the company has its work cut out for it. There’s a multi-layered
community of intermediaries between the stock and the people who want to buy it, and the
company must work the hive to find the most powerful sneezers able to spread the word
about the stock.
The first stop is the market analysts who cover the stock. Once again, the marketplace sees
this group as being powerful sneezers (when one analyst recommended Amazon.com, the
price of the stock doubled in just a few days). By courting the analyst community, a
company can find a way to communicate the story they’ve created around the stock.
Don’t underestimate the power of the story. There are almost no other cues available to
persuade someone to spread the word about a stock. You can’t see it or touch it or smell
it—it’s just an intangible right to make money in the future. As a result, the story must be
able to describe the reason why the stock is selling for x today but will be selling for 3x
tomorrow.
Brokers are a fascinating component in the spreading of an ideavirus around a stock.
Remember, they’re not paid unless people trade. Buy and hold is the enemy of most
stockbroker compensation schemes, since they only charge for trades and are paid by
commission. Yet, for many decades, brokers were seen as powerful sneezers, especially if they
helped make you money in the past. In fact, they’ve always been promiscuous sneezers,
motivated (whether in the short term or the long term) by their ultimate financial gain. A
“good” broker is one who realizes that if he postpones financial gain in exchange for helping
his clients make money in the long run he’ll get more and more clients.
All this is changing as the world shifts to trading online, and more important, getting stock
news online. Suddenly, anyone can talk about stocks, anyone can post to a bulletin board,
and anyone can spread a rumor.
Unleashing the Ideavirus 140 http://www.ideavirus.com
As a result, stock ideaviruses spread much more often and much faster. In one case, the
public markets knew about a CEO’s plan to quit before his board of directors did. Because
the individuals who post these notes are anonymous and possess unknown motivations, the
chances that they’ll develop into powerful sneezers is slight. But the sheer number of posts
(more than 100,000 a day on Yahoo!’s bulletin boards alone) means that they have influence.
An astute CFO or CEO can look at the key factors in the creation and spread of a stock
ideavirus and launch a campaign to move the virus with a velocity and vector they’re
comfortable with, and more important, aim it at the appropriate hive.
Note, for example, that some stocks, like Iomega, are the darlings of online stock bulletin
boards. As a direct result, those stocks are far more volatile than the average. Live by the
sword…
Yahoo! has worked hard for years to manage the story about its stock. Gary Valenzuela, the
legendary former CFO at Yahoo!, was obsessed with three things:
1. Become a blue-chip stock, one that institutions would happily own.
2. Become profitable, to distinguish the Yahoo! story from its competitors.
3. Underpromise and overdeliver, always beating the “whisper numbers” that analyst
established for the company’s quarterly earnings.
As a result, Yahoo! stock has consistently and regularly outperformed its competitors. And
due to the success in labeling Yahoo! an Internet blue chip, the stock is much less susceptible
to swings due to rumors.
Was that expensive in the short run? No doubt. When the market was looking for good news
and hype, Yahoo! often refused to deliver. Short term gains were forsaken for building a
story, a story that could become an ideavirus to be delivered by analysts and other powerful
sneezers.
One way to predict the future of a stock, then, is to see beyond the story and understand
whether the company is actively managing the ideavirus, and doing it in a way that will move
it to the right hive.
Unleashing the Ideavirus 141 http://www.ideavirus.com
Bumper Sticker Marketing
Years ago, I was a walking parody of a high-tech yuppie.
I worked as a poorly compensated marketer at a start-up software company in Cambridge,
Massachusetts. I drove a dented old Saab. I used a Macintosh. And on the back of my Saab I
proudly affixed a bumper sticker that read, “I’d rather be driving a Macintosh.”
This is an ancient form of ideavirus marketing, of course. I used my car as an amplifier,
exposing my message to hundreds or thousands of people. But even better, given the
neighborhood I was driving in, I was focusing the message on an appropriate hive, and given
the car I was driving, adding power to my sneezing.
Think about it. If the bumper sticker had been on the back of a junker Chevy, rusting
outside an abandoned farm in Oklahoma, you would have had a very different response to
the message, no?
The neatest thing about this technique is the way Apple converted the private (what sort of
computer do I use) to the public (my proclamation of how happy I was to be a Mac user).
There are countless opportunities for marketers to do precisely the same thing today. And
not just on your bumper. Some marketers ride along with their product—the Ralph Lauren
pony, for example, is advertising on the front of your shirt all day long. Others manage to
make it a more political choice—Marlboro, for example, was one of the ten largest marketers
of imprinted clothing a few years ago.
Picking the medium for your “bumper sticker” is important, but it’s just as important to
determine why someone is going to be willing to stick his neck out to promote your product.
Personal pride is an excellent tactic! If people are willing to sneeze on your behalf because
they’re proud of you, your product and their association with it, you’re in. Now all you’ve
got to do is give them a smooth way to spread the word.
Unleashing the Ideavirus 142 http://www.ideavirus.com
No, You Go First!
The challenge of the new idea is that very few people want to go first. Who was the first to
swim in the Charles River in Cambridge, Massachusetts after years of it being off limits for
health reasons? Who was the first to give their kid the chicken pox vaccine? Which company
chose to be the first to file its taxes electronically?
One of the key reasons to launch an ideavirus is that you can give people a risk-free, cost-free
way to check out the safety of your idea before they commit. And more important, you can
create an aura around your idea—an aura of inevitability, of invincibility. When everyone is
buzzing about a new technique, tactic, service, musical style, club, food—whatever—it’s far
easier to put fear aside and try it.
But just as people are hesitant to be the first to buy a fancy new product, many are hesitant
to try a fancy new idea. There are plenty of people who want nothing to do with a new song
or a new book… they’re happy to wait until it’s been screened, filtered and accepted by the
mainstream.
So, depending on the hive you choose, you need to make it clear to that consumer that your
idea has arrived. That the water’s warm, the air is safe to breathe and your idea is a
comfortable, tried and true one.
One way to do this is with bestseller lists. And with testimonials. And by exposing the digital
word of mouth record to let them see the countless people who have tried it and liked it. Do
it with the specific objective of reminding people that others have taken the risk and happily
survived. If you work at this and do it on purpose, you’ll be amazed at how much water you
can drain from the river—how easy it is to bring the rocks to the surface, how powerful you
can make the message when you expose the connections that led you from person A to
person B. It’s in this sort of active ideavirus marketing that many brands are able to run rings
around the competition.
Even before you do this, offering your idea in trial-sized, bite-sized portions is critical. Many
companies have created neat, effective online products, only to see them fail because they
Unleashing the Ideavirus 143 http://www.ideavirus.com
required consumers to go through a time-consuming download before they could use
them… and if they couldn’t use it, they couldn’t understand why they wanted it! Catch-22:
a product you don’t know if you want to download until you download it.
Give them a version instead that doesn’t require a download and doesn’t work as well—but
still makes their life better. Why? Because now that I’ve sampled it without risking a virus or
taking a lot of time or trying to understand the arcane intricacies of downloading in
Windows, now I’m willing to invest the time to do it.
Unleashing the Ideavirus 144 http://www.ideavirus.com
Digital Media Wants to Be Free
When was the last time you bought some table salt?
Odds are, you didn’t pay very much. Salt is cheap. Why? Because once you own a salt mine
and pay for a salt factory, the cost of making a pound of salt is low indeed. But because
there’s more than one salt mine out there, the competition for getting salt sales is pretty
intense. And given that all salt is pretty much the same, why pay more?
Pricing battles are certainly not unusual in physical goods. In fact, almost every competitive
category of item that’s entirely physical (without an idea attached) uses cost-based pricing. In
other words, it’s a commodity. When those rules are abandoned (as they were with crude oil
during the Arab oil embargo) consumers are shocked and angry.
For a number of reasons, this pricing approach hasn’t really kicked in with intellectual
property. It only costs McKinsey a few hundred bucks to write a report for Chrysler, but
they happily charge a few million dollars for it. One more copy of a Bob Dylan CD only
costs 80 cents to make (less than a vinyl record!) but it sells for twenty times that.
Why?
The biggest reason is that intellectual property is rarely a commodity. There are many kinds
of salt, but there’s only one Bob Dylan. And when you want to listen to Dylan, it’s not clear
that 10,000 Maniacs is an acceptable substitute.
Because intellectual property is unique, it has long resisted a trend toward commodity
pricing at the margin. In fact, the price of most forms of intellectual property has increased.
Barring one big exception:
Stuff that went from being expensive to being free.
The most popular web server software (the programs they use to run most giant websites) is
not sold by Microsoft. And it doesn’t cost $10,000. It’s free.
Unleashing the Ideavirus 145 http://www.ideavirus.com
The most popular web browsers are free.
The cost of listening to a Beethoven concerto went from $30 (at some fancy theater in
London) to $0 after radio was invented.
The cost of watching a movie on network television is zero.
The mathematics of the ideavirus make it too compelling for the creators of viruses to stay
greedy.
The more people know your virus, the more it is worth!
Thus, if charging people for exposure to your virus is going to slow down its spread, give it
away.
Apple just cut the price of WebObjects software from $50,000 a copy to $699. That’s a
98.7% decrease in the price.
Why? Because Apple realized that unless a lot of people use their software, no one will use it.
Take a look at http://www.mp3.com. Pick an obscure music category like Celtic. Go to the end of
the bestseller list: there are 1,168 songs listed. These are not illegal copyright scams, where
the music has been stolen by the artist. These are real songs, posted by the artists or their
labels. The whole song… not part of it.
Why would anyone do this? Give away an entire album of music when Bob Dylan can
charge $16?
Look at it from your point of view. An unknown artist is a commodity. An unknown artist is
the same as a box of salt. If you don’t know why the artist is unique, why pay?
Look at it from the artist’s point of view. The cost of giving away songs is literally zero. Once
you’ve made a record, the cost of one more copy of an MP3 is nothing. And if it helps you
Unleashing the Ideavirus 146 http://www.ideavirus.com
get listened to, if it helps you build your virus, then you’re one step closer to no longer being
a commodity!
In fact, many artists would pay people to listen to their MP3 cuts if they thought it would
help them break through the clutter and get famous. Take a look at the Payola section of
MP3.com. You can do exactly that… pay money to have your song promoted so you can
give it away for free.
Of course, once you’re famous, you can go ahead and charge $16 for your CDs.
Or can you?
Sure, there’s going to be room for collectibles. For live events. For autographed guitars. But
once something is no longer hot and fresh and the latest, rarest thing, why wouldn’t the selfinterested
artist go ahead and give it away free to stoke the ideavirus for the next release? In a
competitive marketplace where there’s transparent information about who’s listening to
what, the Internet becomes radio. And artists know that charging radio stations is dumb.
This same logic applies to books. And to just about any other sort of digital media you can
think of. Unless there’s an extraordinarily unique property of the media being offered, I
maintain that sooner or later it’s going to be free. The Bloomberg machine used by stock
brokers, for example, commanded a huge price premium for years, because the combination
of excellent data and locked-in user interface meant it wasn’t worth switching. But as the
web replicates more and more of the data available, it’s inevitable Bloomberg’s market share
will decrease—and their prices will as well.
The exciting thing is that people who go first, who put their previously expensive digital
media out there for free, will gain the lion’s share of attention and launch bigger and longer
lasting viruses.
So. Who wants to go first? And who wants to go… last?
Unleashing the Ideavirus 147 http://www.ideavirus.com
Van Gogh Lost His Ear To Prove A Point
When Vincent was painting, he often sold his work for just enough money to cover the cost
of paints and canvas. Back then, his ideas and his paintings were one and the same, and
neither was held in very high regard.
Over the last hundred years or so, something has changed. Instead of selling for $200, or
$2,000 or even $20,000, it’s not unusual to read of a $10,000,000 sale of a Van Gogh. Over
time his paintings have increased in value with each sale. But the paintings haven’t changed
at all, have they?
What’s changed is the value of his ideas and the popularity of his ideas—not the ideas
themselves. It’s easy to get a reproduction of a Van Gogh. For a few hundred dollars, you can
even get a painted reproduction that only a trained expert can tell isn’t the original. So why
pay twenty million dollars?
Because you’re buying a souvenir. An expensive souvenir, no doubt, but a souvenir
nonetheless. The original painting is a priceless keepsake that reminds you of the idea
Vincent Van Gogh first unleashed on the world. And unfortunately for Van Gogh and his
heirs, it took far too long for the ideavirus to spread.
Compare this inexorable and dramatic increase in value with the resale value of a newspaper.
Today’s newspaper is “worth” fifty cents to a dollar. The combination of recent news and
events in one handy packet makes it a reasonable purchase. However, yesterday’s paper is
virtually worthless. And if you’ve got a big stack of them, you’re going to have to pay me to
take them away.
Why? What happened? Simple: the newspaper is a vessel for ideas with very short half-lives,
and once the ideas aren’t fresh any more, they’re worthless. Imagine, though, how much you
could sell tomorrow’s paper for—especially if you sold it while the stock market was still
open.
Unleashing the Ideavirus 148 http://www.ideavirus.com
This is a lesson in one way to make your digital media valuable: keep it fresh. It’s getting
harder and harder to do; they used to send Charles Dickens’ serialized novels over here by
boat—news that was three weeks old was considered fresh—but that doesn’t mean you can’t
succeed.
By focusing on souvenirs and speed, creators of digital media can create two effective ways to
profit when we play by the coming new set of rules.
Unleashing the Ideavirus 149 http://www.ideavirus.com
Answering InaÕs Question
So how is a bookstore to make money? Or a publisher? Or an art dealer or a consultant or a
music label?
The biggest objection to ideavirus thinking is that it represents a substantial change from
standard operating procedures. Successful companies are in no hurry to rock the boat…
especially if it represents a significant change in the status quo and a risk to planned-for
revenue and profits.
Mighty Words (an Internet articles publisher) is aggressively targeting traditional book
publishers and re-sellers by creating a new online business that cuts out all the middlemen
and lets authors sell works (preferably 15 to 60 pages) directly to readers. Go to their site and
you can find thousands of articles, priced from $3 to $30 each.
Mighty Words gets half the revenue, the author gets half, the reader gets insight and wisdom
and everybody wins. By creating new markets for mid-length ideas, the company seems to be
filling a niche. Of course, then they can turn their success into dominance by integrating up
the food chain until they disrupt all the competition in the publishing world and profit
mightily.
So you’d think that the concept of ideaviruses would be attractive to this maverick company.
After all, they’re only a few months old.
Not true. It bugs them terribly to give away ideas, because it flies in the face of their brand
new business model. After all, if an author profits most by giving away her work, how does
Mighty Words make money?
If you catch yourself asking this question about a new business model innovation (“How
would we make money?”) you’re headed for trouble. The Internet doesn’t care how you
make money. The Internet isn’t going to wait while you figure out how to react. Instead,
there’s some crazy entrepreneur who’s willing to spend years of his life making you miserable
by wrecking your business model.
Unleashing the Ideavirus 150 http://www.ideavirus.com
Email didn’t ask the fax companies if it was okay with them if a new, instant, permanent,
digital communications tool came along and wrecked the fax business. Matchmaker.com
didn’t hold meetings with the extremely profitable video dating services out there to find out
if it was okay for them to launch. Who cares if Matchmaker.com never makes money? What
matters to the existing businesses is that these new kids on the block have wrecked the
business landscape for the old providers.
Giving digital media away is a reality. Non-dominant players in any industry will always
succeed more by giving away digital content and then profiting later than they will by
holding back to preserve somebody else’s business model.
It was a mistake for the record companies to fight radio and MTV. It’s a mistake for them to
fight Napster. Rather than fighting to patch the leaky bucket, perhaps they could redefine
their roles so they can figure out how to profit from a “free” world.
Unleashing the Ideavirus 151 http://www.ideavirus.com
Crossing The Chasm With An Ideavirus
In his brilliant book, Crossing the Chasm, Geoffrey Moore unleashed a powerful ideavirus
about how new businesses and new ideas get spread. Basically, there’s a chasm in the product
adoption cycle.
The curve used to look like this:
On the left are early adopters, the nerds who love new stuff, who want to get their hands on
anything neat and potentially wonderful. On the right are the laggards, who are still having
trouble getting rid of their steam engine cars.
The meat is obviously in the center. That’s where the vast majority of us live, and where the
combination of big audience and pretty decent pricing is most attractive to a marketer.
In the old days, people believed that you could introduce a product to the early adopters, use
the high profits from those sales to ramp up production and advertising, and then roll the
product out to the masses.
There’s a problem with this view: there’s a gap in the curve. A chasm.
Unleashing the Ideavirus 152 http://www.ideavirus.com
What happened? Turns out people on the right side of the chasm aren’t just lazier or less
intellectually curious than the folks on the left of the chasm. It turns out that people on the
right are fundamentally different from the folks on the left. How?
Pre-chasm people want something cool. Post-chasm people want something that works.
A nerd wants the latest personal digital assistant. An executive wants to keep her
appointments straight.
A cutting-edge IT guy at Allstate wants a device that will use satellite technology to update
claims instantly. The CEO at Nationwide wants something that will reduce costs.
A fashionista wants the latest haute couture outfit from Paris, regardless of how ridiculous it
looks. The party girl wants something that’ll get her a hot date next week.
The foodie wants maple-jalapeño corncakes, layered with crème fraiche and bourbon. The
hungry person just wants something to eat.
As you can see, focusing on the folks who will give you early feedback, be your initial
sneezers, your first customers and probably your start-up’s employees is a one-way ticket to
doom. Their advice will help you make stuff that’s expensive, heavy, hard to use, awkward
Unleashing the Ideavirus 153 http://www.ideavirus.com
and difficult to understand. You’ll be the darling of some well-respected sneezers, and then
you’ll fail. This is why many ideaviruses start with plenty of powerful sneezers but end up
dying.
It happened to Apple with the Newton. It happened to Microsoft with almost every product
they’ve ever launched (Bill Gates is to the far, far left of the chasm—that’s why it takes
Microsoft to version 3 to build something that catches on). It happened to Reebok and to
Stephen Sondheim and to Lou Reed. In every case, they indulged the pre-chasm audience
and lost the big wins on the right. (Of course, in some cases—like Microsoft—sheer staying
power is able to force you over the chasm.)
The challenge in launching an ideavirus is to understand who the pre-chasm sneezers are,
and using them but not letting them use you. In other words, they’re the ones who are most
likely to embrace your new idea and talk about it, but if you don’t get past them to the rest
of the curve, you’re doomed.
Why do Woody Allen movies consistently sell so few tickets? They’re certainly adored by
critics, nominated for awards and attended by a core group of sneezers. The reason is simple:
the virus hits a chasm. There’s a huge gap between the Woody Allen audience and the rest of
the population. Because of this chasm, the word rarely spreads as far and as wide as it could.
The success of his latest movie, “Small Time Crooks,” points to the problem. This movie has
box-office results that rank among the top four he’s ever released for one simple reason.
During the month it was released, it was the only clean family movie available. By focusing
(intentionally or not) on creating a wry, funny movie that was understandable at many levels
and worth bringing your kids to, Woody crossed the chasm. Suddenly, the sneezers were
saying, “This is a great movie for your family,” instead of saying, “this is another great
Woody Allen movie.”
Some viruses are just never going to cross the chasm. Try as they might, the computer nerds
are having no luck at all getting normal people to start using Linux. And the guys who sell
the hottest hot sauce in the world are just not going to find themselves on the table at TGI
Friday’s restaurant.
Unleashing the Ideavirus 154 http://www.ideavirus.com
But that’s okay. It’s okay because these idea merchants understand that the hive they’re
targeting is not everyone. They understand that if they choose the right hive, it’s okay if it’s
small, it’s okay if it’s not everyone. The caveat, of course, is to match your expenses and your
expectations to the size of the hive you’ve chosen. If you spend big on product development
and marketing, figuring that will get you over the chasm, it better.
Unleashing the Ideavirus 155 http://www.ideavirus.com
The Myth Of The Tipping Point
One of the most seductive ideas in Gladwell’s The Tipping Point is that somehow a magic
moment appears when the entire population goes from blissful unawareness of your offering
to total and complete infatuation.
While this certainly appears to happen, it’s not a reality for most companies and most ideas,
and it’s not even a requisite for mindblowing success. There are two related reasons for this.
The first is that it ignores the power of the hive. The chances that you’re going to launch an
ideavirus that consumes the entire population is slim indeed. After all, there are seven billion
people out there, and all of them have very different needs and communication cycles. Even
if you just boil it down to the United States, or to Republicans with Internet access, it’s
pretty clear that large hives very rarely tip about anything.
The second reason is that winning and tipping aren’t the same thing. In order to really win
with an ideavirus, you have to concentrate your message very tightly on a specific hive. But
even then it’s not clear to me that you have to tip to win.
Let’s take a look at eBay, for example. By almost any measure, eBay is a winner. It’s
employees are millionaires and billionaires. Early investors are delighted. Users are happy,
with time spent on the service going up all the time.
But has eBay tipped? Certainly not in terms of awareness among the general population.
When asked to name an online service, only a tiny fraction of the population picks eBay as
their first choice. But it gets even more obvious when you ask people where they go to buy
and sell used junk. The vast majority of people are using classified ads and garage sales, not
eBay.
Yes, the management of eBay is on the cover of Fortune and Business Week at least once a
month, or so it seems. Yes, every meeting at certain high-tech companies includes the
sentence, “But will this allow us to become the eBay of [insert business here].” Within a very
small, very focused, very profitable hive, eBay is a winner. But it didn’t happen because some
Unleashing the Ideavirus 156 http://www.ideavirus.com
magical tipping process took place. It happened because a smart, focused, powerful ideavirus
started and spread across a concentrated hive of investors and pundits, and this led a tiny
company to have a huge stock market valuation.
The reason I point out this myth is that it’s dangerous. Dangerous because it leads idea
merchants to believe that if they just wait long enough, something will happen and make
them tip—like Yahoo! or the Atkins diet or Nike or the Macarena. I don’t buy it. The odds
are with you if you focus on small hives, filled with pre-chasm sneezers, and then obsess with
crossing the chasm as fast as you possibly can. If you tip, that’s a bonus.
Unleashing the Ideavirus 157 http://www.ideavirus.com
The Compounding Effect
One of the factors that makes the tipping point myth seem more real is the power that comes
from multiple sneezers. While one or two recommendations might make for a smooth
transition, there’s no doubt that as the number of powerful sneezers recommending an idea
to you increases, the chances that you’re going to use it dramatically increases.
This is a genuine side effect of the tipping point. As you are surrounded by hive members
who loudly sneeze about a new idea, the greater your chances of at least trying the idea.
Rather than decreasing returns, as we find in advertising, there are actually increasing returns
from an ideavirus. The more people who have it, the more you want it.
Are there iconoclasts who fight every trend? Of course. They wouldn’t be seen in a hip car or
a hip restaurant or listening to a pop tune. But for most individuals, in most hives, the
compounding effect is quite strong.
Thus, one of the most essential tasks an idea merchant can accomplish is to bring all positive
news to the forefront. They make every hive member think that every other hive member is
already converted to the virus, thus creating the self-fulfilling prophecy that leads to success.
Publishing houses do this when they print lots and lots of copies of a book and ship it out to
stores. If there are tons stacked up by the cash register, many people think that this must be
the hot new book, so they buy it. On the basis of this trial, the book shows up on the
bestseller lists soon after being published. This, of course, leads to more people trying it,
because, after all, it’s on the bestseller list. So, without any genuine “word of mouth,” the
book has established a much larger foundation. It won’t get any bigger unless the idea is
virusworthy, but at least the book got a shot.
On Eric Raymond’s page promoting his essay “The Cathedral and the Bazaar,” he lists and
points to critiques of his work. Why? Because bringing these critiques (both positive and
negative) to the forefront is an excellent way to bring the compounding effect into play.
Unleashing the Ideavirus 158 http://www.ideavirus.com
Most marketers focus on getting organic word of mouth going without taking the time to lay
a framework for the compounding effect. Music Direct, on the other hand, goes to great
lengths to leverage powerful sneezers. On their site (www.amusicdirect.com) they list the
recommended recordings of several high-end stereo magazines. Each one is linked directly to
their online ordering service. Thus, you can read a review in Stereophile and know that you’re
only a click away from buying it on their site. Look at a few of the lists and you’ll notice that
the same record shows up more than once. Boom. Even if you weren’t considering buying
that title, the fact that three trusted sneezers have recommended it makes it much more likely
that you’ll consider it.
The folks at Telarc Records learned this lesson early on. Unable to compete with the big
boys at the other classical music labels, they recorded the Cincinnati Orchestra playing
dramatic renditions of songs that only a stereo lover could love. Big cymbals. Cannons. You
get the idea.
Then, they worked hard to get high-end stereo shops to use the CDs they were recording to
demonstrate their equipment. Thousands of consumers who might never have rushed out to
buy another recording of Tchaikovsky’s “1812 Overture” now discovered that Telarc’s
recording was being used anytime they listened to $5,000 speakers or $3,000 amplifiers.
Hey, if you were willing to drop 20 large on a stereo system, certainly it was worth a few
more bucks to have the best CDs to play on it, wasn’t it?
Unleashing the Ideavirus 159 http://www.ideavirus.com
Bill GatesÕ Biggest Nightmare
One of the repeated mantras during the Microsoft anti-trust sideshow was that middleware
threatened the very essence of Microsoft’s cash cow: the Windows OS.
Basically, middleware is software that sits on top of the operating system on your computer
and talks to the Internet or other programs. Once you develop a killer piece of middleware, it
doesn’t matter what operating system you’re running—the middleware works the same. The
first successful example of middleware was the browser, but you can be sure there will be
more.
Today I spoke to a woman named Louise Wannier who developed a piece of software called
enfish. You can find it at http://www.enfish.com.
What if there were a piece of middleware that was designed for people who had an “always
on” connection to the Net. And what if that software let you automatically track your stocks,
your email, your calendar, your instant messages—all the stuff you spend time doing online,
but in an organized way, and all at once?
If you’re like me, that accounts for the vast majority of time you use the computer.
Suddenly, Windows is obsolete.
Sounds like it’s time for Louise to start shopping for a new Porsche, no?
But there are some problems. And all of them are related to the idea she’s created and how to
turn it into an ideavirus.
Problem #1 In order to use enfish, you have to download code. Experience has shown us that
this is a huge amount of friction with which to saddle a new idea. Basically, you can’t enjoy
the software until you go through the pain and suffering of downloading and installing it.
Unleashing the Ideavirus 160 http://www.ideavirus.com
Products like Shockwave and various forms of wallets have shown us that it can cost as much
as $100 in direct to consumer marketing expenditures to get someone to download a piece of
software. In the case of enfish, this is way, way too much.
Solution: Get rid of the download if possible. If not, make it swift and painless.
Problem #2: This is a private experience. Unlike ICQ or Hotmail, which are both based on
communications and are thus pretty viral, enfish saves you time by organizing your life and
your data, and so you’re not naturally inclined to spread the idea. In other words, it doesn’t
do its own sneezing, nor does it reward you for sneezing on its behalf.
Solution: Make it public. Let people post their bookmarks and layouts for their co-workers.
Figure out how to turn it into a communications tool because communications tools are the
most likely to go viral.
Problem #3: It’s not very smooth. It’s awfully difficult to describe what enfish does, because
it’s not simple. It’s biggest strength—that it solves a problem you didn’t know you had—is
also a huge hassle when it comes to marketing the thing. “Free Email” is smooth indeed.
“Automated organizer for always-on Internet knowledge workers that saves you three hours a
day” is not.
Solution: This is the hardest one. Breakthroughs frequently have this problem. Figure out
how to teach the sneezers what to say… even if it means giving them a pre-written email to
forward to friends.
Problem #4: There’s no existing amplifier. There are plenty of sites where people talk about
cars or hobbies or restaurants. Find a hive and you can talk to them. There are magazines
about gardening and starting Internet companies. There are TV shows about cooking and
the weather. But there’s no natural way to amplify a message about the problem that enfish
solves. There are few easily identifiable hives that are just sitting there, waiting to hear from
enfish.
Solution: Use advertising to feature your most satisfied users.
Unleashing the Ideavirus 161 http://www.ideavirus.com
Problem #5: The ideavirus isn’t a natural monopoly. In other words, once they do a great job
of spreading the virus, it’s not clear that enfish’s solution will be the only one to triumph.
One of the amazing things about ICQ, for example, is that the better they did, the better
they did. In other words, there were network effects that created a natural monopoly.
Unfortunately for enfish, there isn’t an obvious reason why an enfish knock-off couldn’t be
as good as enfish.
Solution: The same communication tools that made it go viral will also support its position
as a monopoly.
The good news is that once it catches on, enfish will be extraordinarily persistent. It will sit
on your desk for years, saving you time and making enfish a profit as they go.
The other good news is that because the benefit delivered by enfish is so awesome, once the
virus starts to spread through a hive, it ought to spread with high velocity, and with the
support of the very best kind of powerful sneezers. This is a product that can easily attract
the attention of sneezers on the left side of the chasm (the early adopters) but also offers very
real benefits that will make it fairly smooth to transfer to the right side of the chasm.
So what should enfish do?
My recommendation is that they focus on a single hive: people who trade stocks online.
Why?
Well, the hive is pretty easy to talk to. There are eight or so online brokerage companies who
could all benefit by sneezing about enfish to their best customers. And online traders talk to
each other constantly, meaning that the message can spread through this community with
enormous velocity.
Further, the benefit to online traders is much, much easier to describe, so it’s a lot smoother:
Make more money by trading in a more organized way.
Unleashing the Ideavirus 162 http://www.ideavirus.com
There’s also a vacuum here. Nobody else is offering this value proposition to this audience.
And finally, because online traders tend to be more technically astute, the friction induced by
the download will be less of a barrier.
After infecting the trader hive, will the enfish virus jump to other hives? Perhaps. But in
order to do that, enfish needs to make two significant changes to their product (remember,
the best ideaviruses are integrated right into the product, not tacked on at the end by the
marketing department).
The first change is to create significant benefits to users that derive from enfish’s scale. In
other words, create a network effect so there’s a natural monopoly.
The second change is to create clear and obvious incentives for existing enfish users to
evangelize and bring in new enfish users. These could be simple bribes, but it’s much, much
more effective if the incentives are related to the product—making it work better when you
have more buddies involved.
If they can accomplish these two tricky tasks (so tricky you’ll notice I haven’t even told you
how to do it!), then the odds of the virus jumping from the trader hive to the Net audience
at large increases dramatically.
Unleashing the Ideavirus 163 http://www.ideavirus.com
Hey, Skinny!
One of the most successful books of the last five years has been The Atkins Diet. Dr. Atkins
has sold more than seven million copies of his books…with almost no advertising.
How does a marketing phenomenon like this happen? Conventional marketing wisdom says
that he would need to spend tens or even hundreds of millions of dollars to motivate the one
out of every 40 Americans who has rushed out and bought his book.
The secret to the book’s success is that the diet was virusworthy. Unlike other diets, it really
generates remarkable results in a very short time (let’s leave the health discussion for another
book).
But being virusworthy isn’t enough. It was also smooth. All you had do to tell someone what
diet you were on was say one word, “Atkins.” Because the author became synonymous with
the diet, it was easy to spread.
But the real secret was amplification. Word of mouth could never generate seven million
conversions, not without being amplified.
So what was the amplifier? Your skinnyness! Whenever the diet worked, nosy and proud
friends would ask the dieter, “Hey, skinny! You look great. How’d you do it?” And the dieter
would proudly respond: “Atkins.”
This self-fueling virus saved Atkins millions. And it would never work for transcendental
meditation, St. Johns Wort or reflexology. Nobody is going to notice your inner peace, after
all. Yes, we may be obsessed with the way we look, but it also leads to powerful viruses.
If you doubt the power of this, take a look at all the tattooed kids on the beach.
Unleashing the Ideavirus 164 http://www.ideavirus.com
Get Big Fast? The Mistake So Many Companies MakeÉ
Why was there so much bloodletting among consumer etailers this spring? How did
Boo.com burn through more than a hundred million dollars in start up cash? Why is
Salon.com, arguably one of the most literate sites on the web, floundering?
The answer for almost all these high profile sites is the same: Get Big Fast isn’t always the
right advice.
Remember, an ideavirus adores a vacuum. So many companies, especially those racing to be
the first to fill a vacuum, spend a huge percentage of their funds trying to prime an ideavirus
by buying huge amounts of poorly executed, poor performing interruption advertising.
Big-spending interruption marketers hope the following:
1.
That sheer bulk will make this bad advertising work.
2.
That sheer bulk will scare off the competition.
3.
That an ideavirus will be spawned and they will become instantly and permanently
popular.
4.
That once they are a center of an ideavirus, their truly flawed business model will
magically make sense. Sort of the AOL effect—you can’t be profitable if you’re small and
illogical, but if you’re big and illogical, you can make a fortune from the companies that
pay you because you have a huge market.
They also fear:
1. That someone else will come along and spend more and move faster than them.
2. That if they take their time, the market will realize that their business model is totally
flawed and they won’t be able to get any more funding.
Alas, the pursuit of an ideavirus has confused their analysis. Instead of viewing themselves as
a natural monopoly, as virusworthy, as needing to fill a vacuum, they could have considered
a very different analysis:
Unleashing the Ideavirus 165 http://www.ideavirus.com
1. The ideavirus space for “online merchant” is already filled. It’s filled by Amazon, and to a
lesser, more twisted degree, by eBay and Priceline.
2. Given that the big space is filled, they ought to understand that the virus they’re going to
spread is going to be far smaller and far more quirky. Thus, the win is smaller, but the good
news is that they’ll need far less money to get there.
3. Once you accept the second point, you can realize that growing a virus slowly is actually a
better strategy. Why? Because you get to perfect your business model as you grow, and you
get holistic, organic virus growth, instead of the forced growth a Super Bowl ad brings you.
In other words, you actually get to earn the people who visit your site.
Diamond Organics (www.diamondorganics.com) is following this approach, and it’s
working. Instead of trying to be a category killer and spending tons of money to persuade the
world that their organic vegetable-by-Federal-Express business is a good one, they’re instead
focused on delighting one customer at a time.
By spending little and scaling a lot more slowly, Diamond is able to build serious sneezers,
sneezers who are quite powerful and need little additional inducement to spread the word.
By getting their systems into shape they avoid the pitfalls that struck ToysRUs.com last
Christmas.
But doesn’t this fly in the face of the ideavirus mantra? In many ways it does. It also
challenges the permission marketing idea that once a consumer solves a problem, they’re not
in any hurry to find someone else to solve the same problem, so vendors can achieve lockout.
The problem with implementing the grow-slow strategy is that you might not get the
chance. If you’re a CEO or marketing executive in a new business, you’re subject to the
Catch-22 of rapid business development. You can’t grow (and you can’t get funded) if you
don’t make promises, but those promises might not be able to be kept. And if the promises
aren’t kept (ToysRUs.com failing to ship in time for Christmas) or the promises cost too
much to keep (Boo.com) it doesn’t matter anyway, because you’ll be bust. So most
entrepreneurs make the promises anyway, even though they realize that organic growth is the
better strategy.
Unleashing the Ideavirus 166 http://www.ideavirus.com
So, there has to be a middle ground. And the middle ground that makes the most sense to
me is to not launch a business that can’t sustain an ideavirus. And second, not to force an
ideavirus to happen before the market is ready for it.
My best example is Amazon. My firm belief is that if Jeff Bezos had launched it a year later
or a year earlier, it would never have worked. A year too early and there wouldn’t have been
enough sneezers and the medium wouldn’t have been ready to spread the word. A year too
late and the market would have been so overheated that his promise would have never
broken through the clutter and attracted the attention of sneezers in the first place.
It’s hard for me to imagine how a $50 million marketing campaign is ever appropriate for
any business to launch an ideavirus. If you need to interrupt that many people, you’re doing
something wrong. Sure, you need that much (actually, much more than that) to launch a
brand and to do traditional marketing. But if you’re virusworthy, you generally can do it for
a lot less money than that.
So you need to match the speed of your virus not just with the money you raise but also with
the promises you make to your investors. Yes, Hotmail and Netscape and ICQ and eBay
grew fast, fast, fast. But that doesn’t mean you will. Optimize for the virus and build it into
your company—or expect that it isn’t going to happen.
Unleashing the Ideavirus 167 http://www.ideavirus.com
The Heart Of Viral Marketing
Remember, viral marketing is a special case of the ideavirus where the amplifier for the virus
is built right into the product. And the hot spot for this wonderful self-propagating process is
in communication products.
Let’s take a look at the history of interpersonal business communication over the last 120
years:
Stamps
Telegraph
Telegram
Telephone
Telex
Fax
Conference Calls
Federal Express
Cell Phones
Videoconferencing
The Web
ICQ and Instant Messaging
It’s a pretty extraordinary list. Twenty-five years ago, when I got my first real job, we had no
voice mail, no web pages, no fax machine, no cell phones, no pagers and no email. I
sometimes wonder what we did all day!
So why is there such rapid innovation in this field, when, at the same time, we are still using
precisely the same Qwerty keyboard found on the early typewriters and the same pink “while
you were out” message pads that came with the first phone?
Unleashing the Ideavirus 168 http://www.ideavirus.com
The answer is pretty simple: Each one of these devices creates long-term profits for its
inventor but is spread at a relatively low cost. And the reason it spreads? Because of viral
marketing.
Communication products demand viral marketing because they’re worthless without
someone at the other end. Metcalfe’s law tells us that the value of a network increases with
the square of the number of people using it. So when there are 10 fax machines in the world,
that’s 25 times better than when there were just 2.
Once I buy a communications device, two things happen. First, I become a powerful sneezer,
telling all my friends to buy one so I can send them stuff. And second, provided it’s a tool
that uses an existing channel (like FedEx or Hotmail), every time I send someone a message,
it’s selling the medium.
The story of Post-It notes is so good it ought to be apocryphal but it’s actually true. Nobody
was buying them. 3M was going to cancel the whole program. Then the brand manager of
the product persuaded the secretary of the chairman of 3M to send a case of Post-Its to the
secretaries of the chairmen of the other 499 Fortune 500 companies.
Suddenly, the most powerful sneezers in the most powerful companies in the country were
sending around memos, all containing comments scrawled on Post-Its. It took just a few
months after that for it to become yet another successful business communication device. A
classic ideavirus.
When I was in business school, a classmate spent a year working on a secret project he
wouldn’t tell anyone about. Turns out he was working to launch MCI Mail, the first
commercial email system. It’s a shame he couldn’t tell anyone, because a bunch of us would
have been happy to tell him what we knew, even 20 years ago: An email system isn’t going to
work if there isn’t anyone to send email to!
MCI was charging about $100 to set you up, and another $20 or so a month, plus usage, for
this new service. Big mistake! They inserted friction early in the process, ensuring that people
would never try it, especially so early in the virus’s life.
Unleashing the Ideavirus 169 http://www.ideavirus.com
My idea was that they give MCI Mail, plus a computer to send it with, to 50 people in each
of the top 100 companies in a given industry. FREE. Suddenly, that industry’s leaders would
be communicating with each other fast and frequently. It would change the culture of the
company. The virus would spread. MCI would win.
What’s the lesson? There are two:
3.
If you can somehow convert your idea into a virus that has to do with communication,
it’s much easier to make it go viral. The best sort of communication is an actual
communication tool (like the fax machine or ICQ) but inventing words, new musical
concepts or other ways people communicate goes a long way as well.
4.
Find the powerful sneezers and beg, cajole and bribe them to use your new tool.
Unleashing the Ideavirus 170 http://www.ideavirus.com
The Great Advertising Paradox
Imagine for a second that there was a machine your company could buy. Figure it costs
anywhere from $1 million to $100 million. You’re promised by the salesman that using this
machine can transform your business, dramatically increase sales and profits and turn your
business into a success.
Interested?
What if the salesperson also tells you that companies who don’t buy the machine have a hard
time growing and often languish… and then she points out that one company, Procter &
Gamble, spent more than $2 billion on machines just like this one last year. Interested?
Oh. There’s one caveat. Actually two:
The ongoing output of the machine can’t be measured. You have almost no idea if it’s
working or not—and there’s no guarantee. If it doesn’t work, tough.
Still interested? Well, after those caveats, there’s just one more fact to mention: On average,
the machine only works for about one out of every ten companies that use it. Ninety percent
of the time, the machine fails to work.
By now, you’ve probably figured out that I’m talking about advertising. Mass market
advertising is one of the most puzzling success stories of our economy. Companies spend
billions of dollars to interrupt people with ads they don’t want about products they don’t
need. The ads rarely work. Ads that are created by less than competent ad agencies and
clients almost never work. One day, I’d like to write a book about the worst ads ever run, but
my fear is that it would be too long.
Now, writing off all marketing expenditures because most of the time they don’t work isn’t
the right answer, either. Hence the paradox. You can’t grow without it. But you often can’t
grow with it, either.
Unleashing the Ideavirus 171 http://www.ideavirus.com
So if advertising is such a crap shoot, such a dangerous venture, why do it? Because for the
last 100 years, the single best way to determine whether a company was going to get big or
stay small was to look at its advertising. Time and time again, aggressive companies with
great advertising—regardless of their industry—have managed to make the ads pay and to
grow and become profitable.
So what changed?
A few things. First, the clutter in the marketplace has finally made advertising even less
effective. A threshold has been crossed, and with hundreds of TV channels, thousands of
magazines and literally millions of websites, there’s just too much clutter to reliably interrupt
people. Add to this the “consumerization” of business-to-business sales (with more ads
directed at businesses than ever before) and the explosion of dot-com advertising, and it’s
easy to see that the game is fundamentally different.
So, what should we do about it? Consider the ironic situation that MarchFirst, Inc. finds
itself in. MarchFirst was formed in 2000, the result of a merger between USWeb/CKS,
which does websites and consulting and advertising, and Whittman-Hart, an Internet
consulting firm.
According to the New York Times, MarchFirst wants to launch with a bang, so they’ve
announced a $50,000,000 advertising campaign designed to “cut through the clutter” and to
“get the name out there, to create strong brand awareness,” according to Robert Bernard,
their CEO.
How are they going to do this? By buying full-page ads in newspapers and Internet trade
magazines, by running TV commercials during sporting events, and even running ads in
lifestyle magazines.
Now, be honest. If you’re flipping through a magazine or surfing through channels on TV
and you come across an ad that is based on “the human desire to be first,” will you stop and
pay attention? Will the slogan “a new company for the new economy” make you sit up and
take notice? Will you give up a few minutes of your precious time to read an ad about a
Unleashing the Ideavirus 172 http://www.ideavirus.com
company you’ve never heard of, which solves a problem you probably don’t have? Not
bloody likely.
Surely there’s a better way for this company to spend fifty million dollars. Surely there’s a
more effective way to start a relationship with the 10,000 people who matter to them than
interrupting millions of us over and over and over….
Old-fashioned, hand-crafted, fun-to-make, sorta-fun-to-watch interruption advertising isn’t
going to disappear altogether. But it’s just a matter of time before CEOs and investors start
measuring their ever-increasing ad budgets with the same critical eye they use for every other
insanely expensive investment they make.
Unleashing the Ideavirus 173 http://www.ideavirus.com
Permission: The Missing Ingredient
When Hotmail launched their free email service, they did almost everything right. They
built a product that was worthy of an ideavirus. They made is smooth. They built
amplification right into the product. They approached the right people and started with just
enough push to make the thing take off.
But then they made a huge error.
They forgot to get permission to follow up. They failed to ask their users (the folks who were
infected by the virus) if it was okay to send them an anticipated, personal and relevant email
every week. They didn’t build an asset.
As a result, the Hotmail website has one and only one way to make money. By selling banner
ads. And nobody clicks on banner ads when they’re reading their email. So advertising on the
Hotmail site is super cheap. And probably overpriced.
We’re talking a multi-billion-dollar mistake here. If they had permission to follow up with
20 million people every single week with an email that was filled with useful information and
relevant ads, they could easily sell the slots in this email for a buck a week. That’s a billion
dollars a year in lost revenue, which, using stock market multiples, is a gazillion dollars in
market cap. All because they forgot to ask for permission.
Let’s face it: It’s unlikely that every single idea you come up with is virusworthy. If we’re
going to have to grow our businesses in a reliable, predictable way, it’s unwieldy to have to
depend on an ideavirus catching on every time we want to grow. We still need a way to
communicate with people directly, to do it when we want to, to talk to the marketplace
without a filter.
Advertising used to be the way to do this. But what a waste! What a waste to have to pay a
magazine for an ad to reach a user you already have! You’ve got a pair of Nike sneakers in
your closet. But Nike has to buy an ad to reach you—they don’t have permission or the
ability to talk to you directly.
Unleashing the Ideavirus 174 http://www.ideavirus.com
Same is true with Stevie Wonder. You bought “Innervisions” because you heard it at a
friend’s house, or on the radio, not because you saw an ad. Yet when Stevie comes out with a
new album, his record label has to start all over again, interrupting you using mass media.
Both Stevie and his label waste a huge asset every single time you buy an album. They have
no idea who you are, and worse, they don’t have permission to contact you again.
The challenge of the idea merchant is to turn the virus into an asset. And you turn the virus
into an asset when you ask the user for permission to follow up directly!
This is probably the biggest mistake that ideavirus marketers have made to date. They launch
a virus—a website, a book, a record, a software program, a food—and enjoy the fruits of the
virus while it lasts, but fail to gain a long term asset. And without that asset, they can’t
launch new products or leverage their existing ones without long lag times and the high costs
associated with contacting the users they’ve already converted.
Unleashing the Ideavirus 175 http://www.ideavirus.com
How A Virus And Permission Team Up To Find Aliens
Turns out that the best way to find alien life somewhere in the universe is to listen.
Specifically, to use powerful supercomputers to scan the spectrums for anomalous sounds.
Unfortunately, there isn’t a supercomputer available that’s powerful enough to get the job
done in our lifetime. Which is why the SETI built the largest distributed computer network
in the world. More than 2,000,000 computers are working, in their spare time, to process
these huge chunks of data.
The mechanics of the system are elegant. Whenever your computer has some downtime, a
screensaver pops up, and behind the scenes, your Pentium processor starts cranking through
data that the computer downloads through the web. But what’s really neat is the fact that all
2,000,000 computers in the network signed up without any advertising or financial
inducement.
Instead, the SETI project launched an ideavirus. Word spread among nerds the world over
that they could help find alien intelligence by having their computers participate in the
network. It’s a classic ideavirus, propagated by some powerful sneezers.
The power of the network, though, comes from the fact that they don’t have to relaunch the
thing every week. That it’s incredibly persistent, of course (once you set it up, it stays set up
until you take the initiative to turn it off), but even better, they have permission to
communicate to their users.
This permission is an asset. You can measure it. You can leverage it. You could turn it into
cash if you wanted to.
Let’s take one more look at the sequence:
1. Invent a virusworthy idea.
2. Make it smooth and persistent.
3. Incent powerful sneezers.
4. Get their permission to follow up.
Unleashing the Ideavirus 176 http://www.ideavirus.com
The Art of Creating an Ideavirus
So far, much of what we’ve discussed is science. Mathematical formulas, game theory,
categories of individuals. This is stuff you have to do well to allow your virus to take hold.
And as the understanding of propagating viruses increases, companies will get better and
better at the tactics.
The hard part, though, is building the virus in the first place. The hard part is inventing an
idea that’s so compelling, so ¡Wow! that it spreads through the hive with high velocity,
converting everyone in its path.
How is it that some ideas move so quickly while others just languish? Why did the Apple
Newton fail so badly, while the Palm took off just a few years later?
Caveat: If I knew the answer, I’m not sure I would tell you! To date, no one has come up
with a repeatable formula for creating viruses in a reliable way. There are precious few people
who are serial virus starters.
My hope was that this would be a long chapter, and I could answer your big question about
how. Alas, I don’t know. I know it when I see it, but I fear the rest is art.
Which means you win by trying. And failing. Test, try, fail, measure, evolve, repeat, persist.
It’s old fashioned and hot and dusty and by no means guaranteed to work. Sigh.
Unleashing the Ideavirus 177 http://www.ideavirus.com
Is He Really More Evil Than Satan Himself?
The Google.com search engine is perhaps the most effective and accurate way to search the
web. Why? Because instead of reading every site and trying to understand the content of
every page, Google just reads the links to sites, and selects the pages that plenty of other sites
link to. This way, popular pages rise to the top, and it’s far harder to trick the engine into
pointing to your page by loading up on clever phrases.
Anyway, a few months ago, if you typed, “More Evil Than Satan Himself” into the Google
search engine, the top link it would return was Microsoft.com. Other links that followed
involved mostly Bill Gates.
How did this happen? How was it that enough hackers, nerds and online intelligentsia
building web pages had a strong enough opinion about Bill & Co. that they would go to the
trouble of creating links to Microsoft that used the words like evil and Satan?
Regardless of the dynamics of the virus itself, there’s no question that it’s pervasive, that it
will take years to erase and that it cost Microsoft dearly. By filling the vacuum and creating
an ideavirus of Microsoft as an all-powerful demon, trouncing anyone who came into its
path, the company’s critics brought the Justice Department knocking on its door.
Intel and Cisco have similar market share in the computer space. McDonald’s has similar
impact in the fast food business. There are plenty of companies that could have attracted
attention. But because Microsoft (through its actions—and inaction) spawned a virus, it was
easier for its critics to get the attention of the government. Regardless of your position on the
case, it’s clear that the negative virus (and Microsoft’s actions that reinforced that impression)
affected the judge’s ruling.
Unleashing the Ideavirus 178 http://www.ideavirus.com
Case Study: Why Digimarc Is Going To Fail
Looking at the world through the ideavirus lens makes it easier to prognosticate about a
company’s success or failure. Consider the case of Digimarc.
Digimarc is a fascinating idea. Create a tiny series of dots that can easily be hidden in
magazine ad. Then, if a consumer wants to go to the advertiser’s website, all they have to do
is hold the magazine up to the camera connected to their PC, and Digimarc’s software will
read the dots and automatically take the user to the company’s site.
Charge the advertisers a tiny fee per ad and everyone wins! The magazines win because it
makes their publications more useful. The advertisers win because it creates a direct and
impactful link between the consumer and the ad. And the user wins because she finds special
promotions or discounts on the site… without having to type in a pesky URL.
So why is it going to be an utter failure?
Because there isn’t enough money in the world to turn this into a success, and the shortcut
path of creating an ideavirus isn’t going to happen any time soon either. I know that I’m
going out on a limb here, as this technology has just been featured in Wired and other
magazines and has gotten a lot of press. Still, bear with me….
First, there are few sneezers. The participating magazines have agreed to run full-page ads
promoting the service (if it helps their advertisers, it’s well worth it) but other than that,
who’s going to talk about it?
There are no promiscuous sneezers. No individual is compensated in any way for spreading
the word. There are no powerful sneezers. It’s not such a great, awe-inspiring or even totally
neat thing to do with your computer. There aren’t overwhelming discounts or secret bits of
information, because, after all, if the advertiser was willing to give a discount to a Digimarc
user, he’d probably be willing to give it to everyone, right?
Unleashing the Ideavirus 179 http://www.ideavirus.com
In addition to having a hard time describing why the service might be virusworthy, it’s not
smooth, either. In order to even find out if you like the service, you have to buy a PC camera
($100, plus the hassle of setting it up) as well as download and install a piece of software on
your PC to run the thing.
Once it is set up, it’s not clear if it’s persistent. The incremental benefit of each use of the
service doesn’t appear to go up—you don’t get better and better rewards the more you use it.
So, as the novelty wears off, the likelihood you’ll keep using it and keep sneezing about it is
small indeed.
Finally, they forgot to focus on just one hive. The ads are running in a wide variety of
magazines, targeting a wide variety of users. Because there’s no overwhelming concentration
in just one hive, the odds of the virus popping are small indeed.
So, wise guy, what would you do instead? Well, I’d re-orient the launch from a general massmarket
consumer to a very vertical business-to-business offering. For example, imagine
putting it on the factory floor. Now, instead of a technician having to drop everything and
type in a URL to see a certain page in a manual, he could just hold up the shop manual to
the camera on his already configured PC. Once you can show that it makes an overwhelming
difference in just a few shops, the word can quickly spread across the hive.
If I really wanted to find the consumer market, I’d focus only on the techiest markets (like
the readers of Wired, but I’d create a benefit to promiscuous sneezers within that market.
Rather than creating a flat environment (each ad goes straight to the user), I’d introduce an
email component that rewards the few people who came in at the beginning for emailing
their techno-friendly friends.
My guess is that if Digimarc values the advertising at retail, they’re probably going to spend
$300 for every regular user they get. In order for it to be profitable, my guess is that they
need to get that number down to $3. Problem.
Unleashing the Ideavirus 180 http://www.ideavirus.com
Why Are These Cows Laughing?
If you were in Chicago last summer or lucky enough to walk through Manhattan this June,
you may have noticed a few cows in the street. Actually, hundreds of cows. Big fiberglass
cows—practically life-sized—located on heavily trafficked corners.
The cows cost under $2,000 to make, yet when they’re sold in a charity auction at the end of
the cow invasion, they’ll go for $10,000, $30,000… up to $50,000 a piece.
What happened? How did a $2,000 cow turn into a $50,000 cow?
Well, it helps that the cows are painted by local artists. Some are poignant, some are
whimsical, but they’re all extremely interesting.
However, that doesn’t explain the whole thing. After all, it’s a used cow, having sat out in the
rain and sleet and soot for months. Add to that the fact that the cows are well-designed, but
the artists behind them are by no means famous. In fact, it’s fair to say that in most cases, the
price of the cow will be among the single highest price these artists have ever received at
auction.
A $2,000 cow turns into a $50,000 cow because of amplification. The same cow sitting in a
SOHO gallery wouldn’t be famous. The same cow straight from the artist would just be art,
not a souvenir of a special moment in the history of a city.
Literally hundreds of articles have been written about the cows. But more important, tens of
thousands of conversations have occurred. It’s impossible to walk down the street with a
friend and pass a cow without mentioning it. After all, it’s a cow, just standing there in the
street.
Like all good art, these cows create conversations. But unlike art in an art gallery, these cows
are amplifying the number of conversations. By sitting there. Every day. Calmly. Sort of
like—cows.
Unleashing the Ideavirus 181 http://www.ideavirus.com
As you pass more cows and different cows and provocative cows, your litany of cow lore
increases. Your ability to talk in interesting ways about the cows increases. “Hey, if you think
this cow is cute, wait until you hear about the cow I saw downtown….”
All of which goes to say that street art, performance art, guerrilla marketing performances…
any sort of interruption of our regular routine can lead to a moment of conversation. When
Abby Hoffman and the Yippies dropped dollar bills in the middle of Wall Street during
lunch hour, they generated a virus among the people who were there, which spread to the
media and beyond. By getting people to interact in a way that they weren’t accustomed to,
the Yippies created more impact than they would have if they’d spent five times as much
cash running an ad.
While this sort of interruption of routine is highly amplified, it is by nature not very
persistent. If you keep interrupting the routine, the routine stops being routine and the
interruptions are. If they kept the cows there for years at a time, they’d be boring. If Abby
Hoffman dropped dollar bills every day, people would quickly stop being excited by it.
That’s why the bar for interruption and guerrilla marketers keeps moving. You can’t do what
created buzz yesterday, because there’s no way that’s going to create more buzz today.
Unleashing the Ideavirus 182 http://www.ideavirus.com
Never Drink Alone
Alcohol manufacturers have two spectacular advantages over most marketers. First, there’s a
huge profit margin built in. Second, drinking is a social phenomenon, perfect for spawning
ideaviruses.
Yet, given this natural platform, most distillers are lazy and just buy a huge number of
interruption marketing events—billboards, magazine ads, liquor store displays. They work
sometimes—remember, all vodka is the same, yet people gladly pay double for Absolut.
Most telling of all is the fact that St. Pauli Girl and Becks Light are made on precisely the
same brewery line in Hamburg, yet people will insist that they prefer one over the other.
Despite their successes, though, virtually all of the money spent on liquor advertising is
wasted. Last year, alcohol marketers spent more than a billion dollars advertising their wares,
but you probably can’t even name the top 20 advertisers off the top of your head.
It’s far, far more effective for alcohol manufacturers to focus on advertising to your friends,
not to you, to invest in building viruses that make it more likely that the group will discuss a
brand and eventually order it… or at the very least, admire the person who does.
One of my favorite examples was reportedly created by the brilliant marketer Bob Dorf.
When Dorf was a PR guy, I’m told he was hired by Galliano to turn their obscure liquer into
a nationwide phenomenon. Realizing that there wasn’t enough money in the world to buy
enough “Drink Galliano” billboards, he took a very different tack. He riffed on an invention
by a California bartender named Harvey and decided to popularize the Harvey Wallbanger.
Unleashing the Ideavirus 183 http://www.ideavirus.com
Harvey was a fairly primitive cartoon, a bit better drawn than Kilroy. But he was also a
drink, a drink that coincidentally used a lot of Galliano.
Dorf then set out to teach the newly-counterculture 1970s drinking crowd about this fun
(hey, it was a cartoon) drink. He printed T-shirts, taught bartenders how to mix the drink
and even sent people into popular bars and had them order the drinks (loudly).
The result was an ideavirus. When one fashion-forward powerful sneezer in a group ordered
one, he’d have to stop and explain to everyone else in the group what it was. That group
took the news to the hive, and the virus spread.
The virus wasn’t particularly persistent (from what I’m told, a Harvey Wallbanger wasn’t
that good) but it was extremely smooth. After someone told you about the drink, all you had
to do to get one was say, “I’ll have one too, please.” High velocity, the virus did exactly what
Galliano had asked for… it put the drink on the radar screen.
Unleashing the Ideavirus 184 http://www.ideavirus.com
The Power Of Parody
The sequel to Mission: Impossible had a huge opening this summer. People talked about the
trailer, and more important, told their friends to go see the movie after they’d been.
But how was Warner going to encourage people to see it two or three or four times? How to
get to the hive of media-friendly, time-wasting teenagers just sitting around looking for ways
to spend money? Most important, how could they cost-effectively remind people that MI:2
was out there and worth seeing again?
They decided to unleash an ideavirus that parodied their own movie.
Mission: Imp is a five minute long web film designed to go viral. It features almost famous
Hollywood stars, better than usual production values (for a web virus) and best of all, a “send
to a friend” button.
Unfortunately, it’s not very funny—so while the foundation is there, it’s not as virusworthy
as it might have been if it were made by someone who wasn’t nervous about offending Tom
Cruise. Either way, though, it’s a smart and aggressive way to get out there and start a virus
to keep a product in the public eye.
Unleashing the Ideavirus 185 http://www.ideavirus.com
Bee Stings And The Measles
My friend Kate was on a canoe trip in Algonquin Park and was lucky enough to find an
outhouse on a deserted island. Relishing the chance to relieve herself without having to dig a
hole in the woods, she rushed in and sat down.
Bad news for Kate, there was a beehive inside. Forty stings later, she found herself sitting in
the lake, waiting for the pain to subside. After a long paddle back to base camp, she got
herself to a doctor. The good news is that after a little pain, she was back to normal.
Unfortunately, she’s now extremely sensitive to bee stings, and has to be ever vigilant, lest she
develop an allergic reaction.
Compare that to the childhood ritual of getting the measles. You get the measles, you sit
through a week of annoying itching, and then you’re done. You’re never going to get the
measles again. You’re immune.
In the first case, exposure to an invading poison led the body to become sensitized. In the
second, it led to immunity. Your ideavirus might behave in either way.
Yes, in general, the ideavirus adores a vacuum. It will spread faster and farther when no
similar virus has preceded it. The idea that you can follow a leader to great success might
work in the old economy (like Schick in razors or Burger King in fast food) but it clearly
isn’t a winning strategy in the new one.
But here’s the interesting special case: Sometimes, after being sensitized by one ideavirus, the
market is more susceptible to a new one. The failure of the Newton, for example, made early
adopters and sneezers more aware of the PDA concept, and it paved the way for the Palm to
succeed. The second Thai restaurant in a given town is more likely to turn a profit than the
first one. Michael Jordan wasn’t the first basketball hero by any means, but our desire to have
a hero, as sparked by earlier stars like Wilt Chamberlain and Larry Bird, made it easier for
Michael to walk in and fill a role that had to be filled by someone.
Unleashing the Ideavirus 186 http://www.ideavirus.com
But IsnÕt It Obvious?
One of the big challenges I faced with Permission Marketing and now with Unleashing the
Ideavirus is that a lot of stuff in these books seems pretty obvious. It’s obvious that marketing
to people who want to be marketed to is more effective than interrupting people who hate
you. It’s obvious that word of mouth is more powerful than ads. It’s obvious that the winner
takes all online. It’s obvious….
But precisely because it’s so obvious, it needs to be written about. Defined. Measured.
Because it’s so obvious, it’s easy to fall into a 100-year-old habit and start doing business the
old-fashioned (expensive but easy) way.
After all, if ideavirus marketing is so obvious, why does eToys need to raise $100 million in
venture capital to pay for old-fashioned advertising? Why are the TV networks having their
best year ever in advertising revenues? Why do really smart businesses suddenly turn stupid
when faced with ad opportunities like Planetfeedback.com?
Because to embrace ideavirus marketing techniques you also have to accept a change from
the status quo. And many of the executives who are now in charge made their way to the top
by embracing the status quo, not fighting it.
It’s much easier to raise venture money with a plan that says you’re going to spend $30
million or $60 million dollars on traditional advertising than it is with a plan that says you’ll
only spend $3 million but employ elegant but difficult techniques to get the word out.
It’s much easier to run the marketing department of a Fortune 500 company around the
command-and-control interruption techniques that got the company there in the first place
than it is to allow the customer to be in charge. And it’s far more difficult to devote your
research and development efforts to building ideaviruses than it is to stick with the
traditional incremental improvements.
Even marketers have heroes. Some kids grow up wanting to be like Sandy Koufax or Bart
Starr. But most of us imagined creating the next great TV commercial or building the next
Unleashing the Ideavirus 187 http://www.ideavirus.com
great brand. We envy the folks who built Coke or Nike or Starbucks or Star Wars. But all
these heroes found their success in a different world—in a factory-based, interruptionfocused
marketing environment.
Today, the world is suddenly very different. Almost without exception, every single win on
the consumer side of the Internet has been due to marketing, and the most effective part of
that marketing is about the ideavirus.
Hotmail, Yahoo!, eBay, Amazon, GeoCities, Broadcast.com, Google—all of them succeeded
because an ideavirus was unleashed and spread.
So, yes, the underlying tenets behind the ideavirus are obvious indeed. But executing against
them, fighting the status quo, getting it right—that’s not obvious at all.
Unleashing the Ideavirus 188 http://www.ideavirus.com
Your CompanyÕs Worst Enemy
She might just work in your office. She’s certainly underpaid. And not very well respected.
I’m talking about the folks who staff your customer service department. Admit it—you and
most of the folks in your company would be delighted if you never heard from or about
these guys and what they do. Their job is to make angry customers go away… quietly.
In the old days, this was a pretty easy job. After all, very few people went to all the trouble to
find your mailing address, get an envelope, get a stamp and write a letter. And if you sent the
writer a coupon good for a few bucks, well that was the end of the story.
Today, it’s very different. Planetfeedback (find them at http://www.planetfeedback.com) makes it
easy for angry customers to find you. And they can carbon-copy their congressman or the
FAA or ten friends.
With digitally augmented word of mouth, an angry customer can leave an online record…
one that lasts for centuries! There’s no statute of limitations online.
Take a look at
http://www.deja.com/products/at_a_glance/glance.xp?PCID=11819&PDID=32765. As I write
this, more than 90 people have ranked Flashcom, a provider of DSL services. Flashcom is
ranked as one of the worst providers of DSL service in the country. Actual comments:
Don’t make this mistake
This is a Mickey Mouse operation. Actually, that’s an insult to Mickey Mouse. Their tech
support is incompetent, their customer service is a front, and their technicians are
useless.
Impossible to get a live person through customer service. Sent over 7 emails and have
gotten back one reply. Had to cancel because of this and they charged me an additional
$150 for early termination. Completely bad attitude.
Unleashing the Ideavirus 189 http://www.ideavirus.com
Fraudulent Thieves
Took my $100 deposit (in October!), didn’t deliver a thing, and won’t give it back (they
deny having any record of it). Their “customer care” people use a wide variety of lies to
string you along. STAY AWAY FROM THESE PREDATORS!!!
Now, Flashcom may be running a first-rate organization. But there’s no way to tell that from
these comments. Question: How many expensive full-page ads will the marketing
department have to run to undo the damage that these public posts are going to do to their
brand for years to come?
Compare those reviews to these for Worldspy.com:
Pound for pound the best…
After trying AltaVista, I had about given up on the notion of a “useable” free ISP. I then
stumbled upon WorldSpy… I’ve never gotten a busy signal through WorldSpy and
consistently connect at 52kbps or higher. I’ve never been disconnected and love the
lack of an ad banner blocking my view.
Great so far
Thanks to all for your reviews that helped me find this service. I imagine it is tempting to
keep something like this a secret in order to keep good service for those in the know. I
know I considered that before I wrote this review! But I felt that as I benefited from
others’ recommendations, I owed it to the Deja community to share my experiences. I
have now been using WorldSpy for a few weeks, and have been pleased with it.
Now, we’re not comparing apples to apples here (Worldspy is free) but that only reinforces
the point. The 290 or more people who posted positive reviews are busy telling all their
friends about this service, spreading the positive news. (NB that Worldspy just tanked. An
ideavirus does you no good if you can’t stay in business!)
Unleashing the Ideavirus 190 http://www.ideavirus.com
Finally, take a look at the reviews for Big Planet. They have more than 1,000 reviews, but it
turns out that many of them are posted by Big Planet affiliates, looking to profit from
bringing on new users.
Thus, we see one ISP on the road to failure because it appears that they’ve refused to invest
any time, money or training in the way they treat customers. We see one that has used a very
different business model (free) and combined it with excellent quality and customer service,
and we see a third that’s busy paying promiscuous sneezers to spread the word. What’s your
company doing?
Instead of putting your weakest people into customer service, what would happen if you put
your best there? Instead of asking for reports on how much pain they’re alleviating, why not
let them tell you about how much joy and delight they’re adding to the customer service
experience?
American Express, ordinarily a terrific, data-driven marketer, has gone in almost entirely the
wrong direction on this issue. Every letter and every phone call is designed to reduce costs,
not to increase personal relationship and delight. And with the amplifying power now
available to sneezers, many companies, on the web and off, will either use this as a weapon or
be the victim of it.
Unleashing the Ideavirus 191 http://www.ideavirus.com
Step By Step, Ideavirus Tactics:
• Make it virusworthy.
If it’s not worth talking about, it won’t get talked about.
• Identify the hive.
You won’t get the full benefit of the ideavirus until you dominate your hive.
• Expose the idea.
Expose it to the right people, and do whatever you need to do to get those people
deep into the experience of the idea as quickly as possible. Pay them if necessary,
especially at the beginning. NEVER charge for exposure if you can help it.
• Figure out what you want the sneezers to say.
You’ve got to decide what you want the sneezers to say to the population. If you
don’t decide, either they’ll decide for you and say something less than optimal, or
they won’t even bother to spend the time.
• Give the sneezers the tools they need to spread the virus.
After you’ve got a potential sneezer, make it easy for him to spread the idea. Give
him a way to send your idea to someone else with one click. Let me join your
affiliate program in sixty seconds or less. Reward the people I spread the virus to,
so I don’t feel guilty for spreading it.
• Once the consumer has volunteered his attention, get permission.
The goal of the ideavirus marketer is to use the virus to get attention, then to
build a more reliable, permanent chain of communication so that further
enhancements and new viruses can be launched faster and more effectively, under
your control this time.
• Amaze your audience so that they will reinforce the virus and keep it growing.
Where are the Cabbage Patch Kids? Why do some viruses burn out more quickly
than others? The simplest reason is that marketers get greedy and forget that a
short-term virus is not the end of the process, it’s the beginning. By nurturing the
attention you receive, you can build a self reinforcing virus that lasts and lasts and
benefits all involved.
• Admit that few viruses last forever. Embrace the lifecycle of the virus.
Cats was a terrific success on Broadway. But even great shows don’t last forever.
By understanding that the needs of the virus change over time (and that the
Unleashing the Ideavirus 192 http://www.ideavirus.com
benefits received change as well) the marketer can match expenditures to the
highly leveraged moments.
Unleashing the Ideavirus 193 http://www.ideavirus.com
The Future Of The Ideavirus: What Happens When Everyone Does It?
Interruption marketing (the kind they do on TV) is doomed to fail, because each marketer
who enters the field has more to gain by adding to the clutter than they do by trying to make
the medium work for everyone else. It’s the classic Hudson River pollution problem—once a
big factory is polluting the river, you might as well too.
Permission marketing, on the other hand, is self-limiting. When people have had enough,
they’ll stop giving permission to marketers, and thus there will be no clutter crisis. Sure,
some folks will cheat by spamming or invading privacy or buying and selling names. But
societal pressure and a few key government regulations should stop the cheaters.
But what about the ideavirus? After it dawns on marketers that it’s working, won’t we all be
flooded by offers to make us promiscuous and an incredible flow of free this and free that?
You bet. I think a few things will occur:
1. The race goes to the swift. Just as Frank Zappa and David Bowie supercharged their
careers by getting on CD early, some marketers will fill vacuums and enjoy profits for years
to come. Latecomers will get hurt.
2. The cost of spreading a virus will increase. The bounties to turn people promiscuous will
increase. The benefit to powerful sneezers will increase. When there’s huge demand for
recommendations, marketers will have to pay more to get them.
3. There will be a significant benefit to becoming a powerful sneezer. Everyone will want to
be Esther Dyson or Walter Cronkite, because that sort of genuine credential can be turned
into a profit for years and years. Thus, we’ll see fewer institutional efforts and more
individuals (free agents) who figure out that they can profit mightily by spreading their own
viruses (this manifesto is a living example of that technique).
4. It’s going to be noisy and loud and cluttered as we transition, with a few huge winners and
many satisfied marketers who dominate a hive but don’t necessarily tip. After that, once the
various media settle down, an equilibrium will return and (hopefully) the good stuff will win.
Unleashing the Ideavirus 194 http://www.ideavirus.com
Good luck. Tell me how it goes for you! Sethgodin@ideavirus.com
STEAL THIS IDEA!
Here’s what you can do to spread the word about Unleashing the Ideavirus:
1. Send this file to a friend (it’s sort of big, so ask first).
2. Send them a link to http://www.ideavirus.com so they can download it themselves.
3. Visit http://www.fastcompany.com/ideavirus to read the Fast Company article.
4. Buy a copy of the hardcover book at
http://www.amazon.com/exec/obidos/ASIN/0970309902/permissionmarket.
5. Print out as many copies as you like.
Unleashing the Ideavirus 195 http://www.ideavirus.com
Acknowledgments
First, some professional sneezing (you can find all these links at http://www.ideavirus.com as well):
1. If you ever get the chance to have Red Maxwell design something for you, grab it. He’s an
extraordinary talent, and even better, a brilliant project manager and a great friend. You can
reach Red at red@designfactorynet.com.
2. One of the best ways to start and spread an ideavirus is to have your company write a
book about it. Books are still the most leveraged way to get powerful sneezers to understand
your ideas and spread them. And the partners at Lark Productions—Lisa DiMona, Karen
Watts and Robin Dellabough (robinlark@mindspring.com) are among the best I’ve ever
found at turning ideas into books. In the past twelve months, they’ve handled the words of
Kinko’s, the Dalai Lama (who wrote the foreword for Bo Lozoff’s inspiring new book) and
me. How cool is that?
3. If you haven’t been reading Fast Company, don’t panic. You can catch up on what you’ve
missed at http://www.fastcompany.com. In a world of hype and shallowness, you’ll find very little
of either here.
4. Malcolm Gladwell’s book, The Tipping Point, will radically shift your thinking. That’s a
good thing. Find this and his other writings at http://www.gladwell.com.
5. A lot of people haven’t kept up with Tom Peters since they bought his very first book.
Don’t hesitate! I reread his stuff as often as I can. Find it at http://www.tompeters.com.
6. I also recommend four other great writers and thinkers. Chris Meyer co-wrote Blur
among other things, and despite his protestations, is beginning to share my hairline. Jay
Levinson is the original marketing bigthinker, and you’re selling yourself short if you haven’t
picked up his books lately. And finally, Don Peppers and Martha Rogers who continue to
be way ahead of me and everyone else in how they’re deconstructing and reconstructing the
way we think about marketing.
Unleashing the Ideavirus 196 http://www.ideavirus.com
7. The guys at Peanut Press are terrific. If you’ve got a Palm, point your browser to
http://www.peanutpress.com and go get some books for it. Thanks, Mike!
I’d like also like to thank Susan Greenspan Cohen, Bob Dorf, Louise Wannier, Alison
Heisler and the wonderful people at Fast Company (especially the incredible Alan Webber)
for advice, insight and encouragement as I plowed through this manifesto. And thanks to my
role model and friend Lester Wunderman.
Jerry Colonna, Fred Wilson, Bob Greene, Tom Cohen, Seth Goldstein and their friends,
partners and associates at Flatiron Partners have been generous enough to give me a platform
and a lab to mess with a lot of new thinking. They certainly keep me on my toes, and are
nice enough to sit through my endless slide shows. Fred Wilson and Tom Cohen, though,
deserve extra thanks. Without the two of them, my internet company would have never
thrived, and you wouldn’t be reading this book. Steve Kane and Stu Roseman are, amazingly
enough, about to throw themselves into this maelstrom. Can’t wait.
Thanks to Don Epstein and David Evenchick at the Greater Talent Network in New York
City for believing in me and then being true to their word and keeping me busy.
For the last year, two people have done everything to keep things in perspective for me…
Lisa Gansky and my Dad. Thanks, guys.
Of course, as always, the real joy in my life comes from my wife Helene and our little
entrepreneurs, Alex and Max.
Unleashing the Ideavirus 197 http://www.ideavirus.com
Script Convert-WindowsImage.ps1 – WIM2VHD for Windows 8!
Whats Coming in SharePoint 2013 — Redmond Developer News
What’s Coming in SharePoint 2013
Many SharePoint 2013 features are already available for testing in the Office 365 previews.
Microsoft offered more details on the improvements in its SharePoint collaboration platform, which is slated to be released as a service and a server in 2013.
Many SharePoint 2013 features are already available for testing in the Office 365 previews, which the company released last month.
Jared Spataro, senior director for SharePoint product management at Microsoft, offered a quick tour of some highlights in the new SharePoint, during a demonstration for technology reviewers on Tuesday. According to Spataro, the underlying architecture hasn’t changed very much with this release, but the company has added some upgrades and I/O performance improvements.
The user experience has also been updated. Users of the new SharePoint will see a flat, spare “modern” UI, with a narrow menu bar at the top.
Microsoft also highlighted the collaboration and social networking aspects of SharePoint, which are among the product’s top uses, according to a Forrester Research survey. However, nothing was really said during the talk about Microsoft’s Yammer acquisition, which will bring Yammer’s enterprise social networking technologies across SharePoint, Office 365, Microsoft Dynamics and Skype.
Better Tagging
In general, Microsoft has improved the metadata aspects in the new SharePoint, allowing users to tag content while posting, Spataro said. People are considered first-class objects in SharePoint. They can be followed, by SharePoint users, but it’s also possible to follow documents, sites and tags across a Web site. SharePoint also points users to content based on a “suggested sites” feature. Users can hover over user profiles and get access to their contact cards, which bring in profile information, including info from LinkedIn and Facebook social networking pages.
My Site in the new SharePoint has its functionality split into three hubs: Newsfeed, SkyDrive Pro and Sites, as explained in this SharePoint team blog. The Newsfeed application has a sort of Facebook-like appearance with photos and e-mail threads, as well as a “Like” button. The Sites application is a tracker of site locations that are important to the user. SkyDrive Pro is a cloud-based storage place for files that works with SharePoint. Spataro said that SkyDrive Pro replaces what used to be called “SharePoint Documents” or “My Documents” in earlier editions, adding that it had been renamed to highlight Microsoft’s investments in consumer cloud storage. Microsoft also offers a free SkyDrive service for consumers.
SkyDrive Pro Replacing SharePoint Workspaces
SkyDrive Pro provides storage, synchronization and sharing capabilities for users. Cloud-based apps will sync to the desktop app by just clicking on the desktop. There’s also drag-and-drop file uploading capabilities from the desktop to the browser-based app, and even drag-and-drop capabilities within the browser-based app, which Spataro called “a new modality for people.” He claimed that he works more in Web apps these days because of the richness of the applications. SkyDrive Pro works with various Office Web Apps, such as Word, Excel, PowerPoint and OneNote.
SkyDrive Pro has access to all of the new SharePoint capabilities. “All of the content manageable in SharePoint can be managed in SkyDrive Pro,” Spataro said.
The fate of SharePoint Workspace (previously known as “Groove”) in SharePoint 2013 got cleared up during the Q&A session. Groove was the invention of former Microsoft Chief Software Architect Ray Ozzie, whose retirement was announced in October 2010.
“SharePoint Workspace was the way that we did document sync and offline access with the previous version of SharePoint,” Spataro explained. “And in fact, SharePoint Workspace was the evolution of the Groove client. The document store, share and sync capabilities that I showed you are actually based on the next evolution of that SharePoint Workspace. We actually used the underlying component that came from Groove, part of the sync engine, to do it. And going forward, our strategy will be to focus on SkyDrive Pro that I showed you today.”
He added that people can still get access to SharePoint Workspace, “but our go-forward strategy investment will be focused on SkyDrive Pro.”
Team Site Gets OneNote
The Team Site is considered the “center of gravity for people in SharePoint,” Spataro said, and Microsoft has enhanced it by adding a centralized OneNote built into it. OneNote is a Microsoft Office application that lets users store photos and text in a sort of digital scrapbook. The OneNote that’s part of Team Site is capable of synchronization, and it can be viewed on various mobile devices with “the same rich view,” Spataro contended. The Team Site also has its own newsfeed, which will sync up with a user’s main Newsfeed. Users tend to use this newsfeed service as a replacement for e-mail, he added.
Spataro said that the new SharePoint is about task completion and using social interactions to do work, so there’s a My Tasks interface that users can plot against a timeline. It’s capable of drag-and-drop operations, so users can create a list of tasks in Excel and drop the file into tasks lists. It’s possible to edit in real time and assign tasks to different people. The whole timeline is viewable in a “project summary” page. Spataro suggested this approach is an improvement over past SharePoint releases where it was “tough to track things.”
There are some SharePoint business intelligence improvements enabled by SQL Server 2012. For instance, the Excel Web App can be used to display a Power View executive dashboard, which graphically displays data. It allows slice-and-dice operations to be performed using the data or the data can be plotted over time. With these capabilities, Spataro said that Microsoft is bringing together social, task management and business insights.
Lastly, Spataro pointed to the Microsoft partner ecosystem and the extra support users can get though SharePoint apps. The apps are available in the SharePoint library or they can be downloaded from the SharePoint Store.
Spataro said he left out a lot in his presentation, noting that FAST search in the new SharePoint now has e-discovery capabilities across SharePoint, Exchange and Lync file shares. Search was one of the SharePoint features that participants in Forrester’s survey said they least liked. However, it’s apparently improved in the new SharePoint.
An overview of the new SharePoint features was described earlier in this blog post by Jeff Teper, corporate vice president of SharePoint.
About the Author
Kurt Mackie is online news editor, Enterprise Group, at 1105 Media Inc.
• Deployment guide for Microsoft SharePoint 2013
-
Deployment guide for
Microsoft SharePoint 2013
Microsoft Corporation
Published: October 2012
Author: Microsoft Office System and Servers Team (itspdocs@microsoft.com)
- Abstract
This book provides deployment instructions for SharePoint 2013. The audiences for this book include application specialists, line-of-business application specialists, and IT administrators who are ready to deploy SharePoint 2013.
The content in this book is a copy of selected content in the SharePoint 2013 technical library as of the publication date. For the most current content, see the technical library on the web.
This document is provided “as-is.” Information and views expressed in this document, including URL and other Internet website references, may change without notice. You bear the risk of using it.
Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.
© 2012 Microsoft Corporation. All rights reserved.
Microsoft, Access, Active Directory, Backstage, Bing, Excel, Groove, Hotmail, Hyper-V, InfoPath, Internet Explorer, Office 365, OneNote, Outlook, PerformancePoint, PowerPoint, SharePoint, Silverlight, SkyDrive, Visio, Visio Studio, Windows, Windows Live, Windows Mobile, Windows PowerShell, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.
Overview of SharePoint 2013 installation and configuration 1
Installation and configuration 3
Configure settings, services, solutions, and sites 5
Prepare for installation of SharePoint 2013 9
TechNet articles about how to prepare for SharePoint 2013 installation and initial configuration 9
Additional resources about SharePoint 2013 installation and initial configuration 10
Initial deployment administrative and service accounts in SharePoint 2013 11
Account permissions and security settings in SharePoint 2013 14
About account permissions and security settings 14
SharePoint administrative accounts 14
Setup user administrator account 14
SharePoint farm service account 15
SharePoint service application accounts 16
Default content access account 16
Excel Services unattended service account 17
My Sites application pool account 17
Other application pool accounts 18
WSS_CONTENT_APPLICATION_POOLS database role 18
WSS_SHELL_ACCESS database role 19
SP_DATA_ACCESS database role 19
All SharePoint 2013 service accounts 46
Configure SQL Server security for SharePoint 2013 environments 47
Configuring a SQL Server instance to listen on a non-default port 48
Blocking default SQL Server listening ports 49
Configuring Windows Firewall to open manually assigned ports 50
Configuring SQL Server client aliases 50
Install prerequisites for SharePoint 2013 from a network share 52
Installer switches and arguments 52
Download and combine the SharePoint 2013 prerequisites on a file share 53
Install the SharePoint 2013 prerequisites at the command prompt 54
Install the SharePoint 2013 prerequisites by using an arguments file 54
TechNet articles about how to install and configure SharePoint 2013 58
Additional resources about how to install and configure SharePoint 2013 59
Install SharePoint 2013 on a single server with a built-in database 61
Run the Microsoft SharePoint Products Preparation Tool 63
Run the SharePoint Products Configuration Wizard 64
Install SharePoint 2013 on a single server with SQL Server 68
Before you install SharePoint 2013 on a single server 68
Install SharePoint 2013 on a single server 69
Run the Microsoft SharePoint Products Preparation Tool 70
Run the SharePoint Products Configuration Wizard 71
Run the Farm Configuration Wizard 74
Install SharePoint 2013 across multiple servers for a three-tier farm 77
Before you install SharePoint 2013 on multiple servers for a three-tier farm 79
Using the Microsoft SharePoint Products Preparation Tool 79
Public updates and hotfix packages 80
Install SharePoint 2013 on the farm servers 80
Create and configure the farm 81
Add web servers to the farm 83
Install or uninstall language packs for SharePoint 2013 85
About language IDs and language packs 85
Installing language packs on the web and application servers 87
Uninstalling language packs 88
Add web or application servers to farms in SharePoint 2013 89
Before you add a web or application server to a SharePoint farm 89
Install prerequisite software 92
Install the SharePoint software 93
Add the new SharePoint server to the farm 93
Remove a server from a farm in SharePoint 2013 97
Removing a web server or application server from a SharePoint farm 97
Remove a web server or an application server from a farm by using Control Panel 98
Removing a database server from a SharePoint farm 99
Install and configure a virtual environment for SharePoint 2013 103
TechNet articles about SharePoint 2013 virtualization with Hyper-V 103
Additional resources about Hyper-V installation and initial configuration 103
Review the general best practice guidance for virtualization 106
Best practice guidance for virtualization 107
Configure the Hyper-V host computer 108
Install and configure virtual networking 109
Virtual local area networks (VLANs) 110
Network adapters and virtual network switches 111
Create and configure the virtual machines 112
Configure the memory for the virtual machines 112
Configure the processors for the virtual machines 113
Configure the controllers and hard disks for the virtual machines 114
Configure services and general settings 115
TechNet articles about how to configure settings for the server farm 117
Additional resources about how to configure settings for the server farm 119
Configure authentication infrastructure in SharePoint 2013 120
TechNet articles about how to configure authentication infrastructure 120
Configure forms-based authentication for a claims-based web application in SharePoint 2013 122
Phase 2: Configure the Web.Config files for an LDAP membership provider 124
Configure the Central Administration Web.Config file 124
Configure the Security Token Service Web.Config file 126
Configure the new web application Web.Config file 127
Create a new web application that uses forms-based authentication with Windows PowerShell 128
Configure a forms-based authentication web application for Windows Azure autohosted apps 130
Configure SAML-based claims authentication with AD FS in SharePoint 2013 132
Phase 1: Install and configure an AD FS server 133
Phase 2: Configure AD FS with the web application as a relying party 133
Configure AD FS for a relying party 133
Export the token signing certificate 135
Phase 3: Configure SharePoint 2013 to trust AD FS as an identity provider 135
Exporting multiple parent certificates 136
Import a token signing certificate by using Windows PowerShell 136
Define a unique identifier for claims mapping by using Windows PowerShell 138
Create a new authentication provider 138
Associate an existing web application with the AD FS identity provider 139
Create a new web application with the AD FS identity provider 140
Configure server-to-server authentication in SharePoint 2013 141
TechNet articles about how to configure server-to-server authentication 141
Configure server-to-server authentication between SharePoint 2013 farms 142
Configure a SharePoint 2013 trust relationship with another farm 142
Configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013 146
Configure server-to-server authentication between SharePoint 2013 and Lync Server 2013 149
Configure app authentication in SharePoint Server 2013 152
Step 1. Configure the SharePoint Server 2013 app authentication trust 153
Configure SharePoint Server 2013 to trust ACS 153
Configure SharePoint Server 2013 to trust the app 154
Step 2. Register the app with the Application Management service 156
Step 3. Configure app permissions 156
Configure client certificate authentication for SharePoint 2013 157
Configure client certificate authentication 158
Configure availability and recovery solutions for SharePoint 2013 160
Configure SQL Server 2012 AlwaysOn Availability Groups for SharePoint 2013 161
Knowledge and skill requirements 162
SQL Server AlwaysOn Availability Group concepts 162
Windows Server Failover Clustering 164
SharePoint Foundation 2013 and SharePoint Server 2013 164
Detailed steps to configure an AlwaysOn Availability Group for SharePoint 164
Prepare the Windows Server cluster environment 165
Prepare the SQL Server environment 166
Create and configure the availability group 167
About replicas and data synchronization 168
Replica configuration requirements 169
Create and configure the availability group 171
Create the availability group 171
Install and configure SharePoint 2013 172
Add SharePoint databases to the availability group 173
Use failover tests to validate the AlwaysOn installation 174
Monitor the AlwaysOn environment 174
Configure email integration for a SharePoint 2013 farm 176
TechNet articles about email integration 176
Configure incoming email for a SharePoint 2013 farm 178
Install and configure the SMTP service 179
Install IIS 6.0 Management tools 180
Configure the SMTP service 180
Configure incoming email in a basic scenario 181
Configure incoming email in an advanced scenario 181
Prepare your environment for incoming email in an advanced scenario 184
Configure AD DS to be used with Directory Management Service 184
Add an SMTP connector in Microsoft Exchange Server 2010 188
Configure permissions to the email drop folder 189
Configure email drop folder permissions for the logon account for the SharePoint Timer service 190
Are attachments missing from email messages that are sent to a SharePoint document library? 191
Configure outgoing email for a SharePoint 2013 farm 192
Install and configure the SMTP service 193
Configure the SMTP service 194
Configure outgoing email for a farm 195
Configure outgoing email for a specific web application 196
Configure services and service applications in SharePoint 2013 198
TechNet articles about how to configure services for SharePoint 2013 198
Additional resources about how to configure services for SharePoint 2013 200
Configure the Secure Store Service in SharePoint 2013 201
Generate an encryption key 203
Refresh the encryption key 204
Store credentials in Secure Store 204
Create a target application 205
Set credentials for a target application 207
Create and configure a Search service application in SharePoint Server 2013 209
How to create and configure a SharePoint Search service application 209
Step 1: Create accounts that are required for a SharePoint Search service application 210
Step 2: Create a SharePoint Search service application 211
Step 3: Configure the SharePoint Search service application 212
Specify the default content access account 212
Specify the contact email address 213
Create content sources in a SharePoint Search service application 213
Step 4: Configure the SharePoint Search service application topology 214
Create a Search Center site in SharePoint Server 2013 215
Deploy people search in SharePoint Server 2013 218
People search prerequisites 218
Configure My Sites settings 219
Add data for people search 222
Add user profiles to the profile store 222
Add information to My Sites 223
Configure result sources for search in SharePoint Server 2013 225
Levels and permissions for result sources 226
Set a result source as default 228
Create and configure Machine Translation services in SharePoint Server 2013 230
Create a SharePoint Machine Translation service application 231
Database section properties 232
Configure the Machine Translation Service 234
Configure Request Manager in SharePoint Server 2013 238
Windows PowerShell examples to enable routing and throttling 242
Windows PowerShell examples routing target tasks 243
Routing and throttling rules 244
Front-end web server selection 246
Request routing and prioritizing 246
Monitoring and maintenance 247
Configure Business Connectivity Services solutions for SharePoint 2013 249
About Business Connectivity Services installation scenarios 249
Deploy a Business Connectivity Services on-premises solution in SharePoint 2013 251
What these procedures help you deploy 251
How to use these procedures and a roadmap of the procedures 253
On-premises scenario prerequisites 255
How to download and install the AdventureWorks sample database 256
Create a SQL Server user on the AdventureWorks database 258
Start the Business Data Connectivity service 259
Create the Business Data Connectivity service application in SharePoint 2013 260
Create a new Business Data Connectivity Services service application 260
Set permissions on the Business Connectivity Services Metadata Store 262
Configure Secure Store Service for on-premises Business Connectivity Services 264
Create and configure an external content type with SharePoint Designer 2013 267
Define general information 267
Define general and Office behaviors 268
Create a connection to the external data 268
Select a table, view, or routine and Define Operation 268
Map Outlook fields and set up the external item picker control 269
Set the Title field for an external list and complete the external content type 269
Set up permissions to the external content type 271
Create a view of an external list 273
Manage user permissions to the external list 275
Synchronize the external list with Outlook 277
Update customer data offline and refresh it online 278
Configure eDiscovery in SharePoint Server 2013 279
Configure communication between SharePoint Server 2013 and Exchange Server 2013 279
Configure Search to crawl all discoverable content 280
Create an eDiscovery center 281
Configure site mailboxes in SharePoint Server 2013 282
Configure SharePoint for Site Mailboxes in SharePoint Server 2013 283
Install Exchange Web Services API on SharePoint Server 284
Establish OAuth Trust and Service Permissions on SharePoint Server 2013 284
Configure Exchange Server 2013 for Site Mailboxes 293
Establish OAuth Trust and Service Permission on Exchange 293
Table of Error Codes for Reference When Running Configuration Checklist Script 293
Configure Exchange task synchronization in SharePoint Server 2013 297
Configure SharePoint for Task Synchronization in SharePoint Server 2013 298
Install Exchange Web Services API on SharePoint Server 298
Configure Exchange Server 2013 for Task Synchronization 299
Establish OAuth Trust and Service Permission on Exchange 299
Configure social computing features in SharePoint Server 2013 300
TechNet articles about configuring social computing features 300
Additional resources about configuring social computing features 301
Configure My Sites in SharePoint Server 2013 302
User Profile service application and profile synchronization 303
Create a My Site host site collection 303
Add a wildcard inclusion managed path to the web application 304
Connect the web application to service applications 305
Enable self-service site creation for the web application 305
Configure My Site settings for the User Profile service application 307
Enable the User Profile Service Application – Activity Feed Job 309
Configure trusted My Site host locations 310
Configure links to Office client applications 310
Add personalization site links on My Sites 310
Create and configure communities in SharePoint Server 2013 312
Configure microblogging in SharePoint Server 2013 315
TechNet articles about microblogging 315
Configure Following settings in SharePoint Server 2013 316
Configure Following settings for My Sites 316
Manage Feed Cache and Last Modified Time Cache repopulation in SharePoint Server 2013 318
Repopulate the Last Modified Time Cache by using timer jobs in Central Administration 319
Repopulate the Feed Cache and Last Modified Time Cache by using Windows PowerShell cmdlets 320
Manage the Distributed Cache service in SharePoint Server 2013 321
Start and stop the Distributed Cache service 321
Change the memory allocation of the Distributed Cache service 322
Change the memory allocation of the Distributed Cache by using Windows PowerShell 323
Add or remove a server in a Distributed Cache cluster 324
Remove a server from the cache cluster by using a Windows PowerShell 324
Perform a graceful shutdown of the Distributed Cache service 325
Change the service account 325
Enable or disable personal and social features for users or groups in SharePoint Server 2013 327
Enable users or groups to use personal and social features 327
Configure web content management solutions in SharePoint Server 2013 329
Configure cross-site publishing in SharePoint Server 2013 331
Create site collections for cross-site publishing 332
Activate the Cross-Site Collection Publishing feature 332
Create content for authoring sites 332
Create and manage term sets for tagging content on authoring sites 332
Create catalog content by using SharePoint lists 333
Share a library or list as a catalog 334
Make a term set available to other site collections 336
Configure search for cross-site publishing 336
Connect a publishing site to a catalog in SharePoint Server 2013 338
Connect a publishing site to a catalog 339
Configure Search Web Parts in SharePoint Server 2013 342
Add a Content Search Web Part to a page 343
Configure the query for a Content Search Web Part 343
Configure the display templates for the Content Search Web Part 348
Add a Refinement Web Part to a page 348
Configure the Refinement Web Part 349
Change the refiner display name 350
Display refiner counts in a Refinement Web Part 350
Configure the display templates for the Refinement Web Part 351
Add a Taxonomy Refinement Panel Web Part to a page 351
Configure the Taxonomy Refinement Panel Web Part 352
Add a Recommended Items Web Part to a page 352
Configure the Recommended Items Web Part 353
Configure the display templates for the Recommended Items Web Part 355
Configure refiners and faceted navigation in SharePoint Server 2013 356
Enable a managed property as refiner 357
Managed properties that are enabled as refiners by default 357
Enable a managed property as a refiner in SharePoint Central Administration 359
Configure refiners for faceted navigation 360
Enable a term set for faceted navigation 360
Add refiners to a term set 360
Set intervals for refiner values 361
Configure result sources for web content management in SharePoint Server 2013 362
Levels and permissions for result sources 363
Set a result source as default 366
Configure recommendations and usage event types in SharePoint Server 2013 367
Create a custom usage event type 368
Record a custom usage event 369
Record a default usage event 373
Change the level of importance of a usage event type 376
Change the Recent time period for a usage event type 378
Enable and disable the logging of usage events of anonymous users 379
Configure workflow in SharePoint Server 2013 383
Installing and configuring workflow for SharePoint Server 2013 384
Workflow Platform types available in SharePoint Server 2013 385
Install and configure SharePoint Server 2013 386
Install and configure Workflow Manager 386
Configure Workflow Manager to work with the SharePoint Server 2013 farm 386
Installing Workflow Manager certificates in SharePoint Server 2013 391
Install Workflow Manager certificates in SharePoint 391
Create a web application in SharePoint 2013 393
TechNet articles about how to create web applications 393
Create web applications that use classic mode authentication in SharePoint 2013 395
Create a web application that uses classic mode authentication with Windows PowerShell 396
Create claims-based web applications in SharePoint 2013 399
Create a claims-based web application by using Central Administration 400
Create a claims-based web application by using Windows PowerShell 404
Create a classic-mode web application by using Windows PowerShell 405
Configure basic authentication for a claims-based web application in SharePoint 2013 407
Configure IIS to enable basic authentication 408
Configure digest authentication for a claims-based web application in SharePoint 2013 410
Configure IIS to enable digest authentication 411
Install and manage solutions for SharePoint 2013 412
TechNet articles about how to install and manage solutions 412
Additional resources about how to install and manage solutions 413
Install and manage apps for SharePoint 2013 414
Downloadable resources about apps for SharePoint 414
TechNet articles about apps for SharePoint 414
Additional resources about apps for SharePoint 415
Overview of apps for SharePoint 2013 417
Where are apps for SharePoint hosted? 417
How are apps for SharePoint and SharePoint sites related? 418
What is the URL for an app for SharePoint? 419
Use and benefits of apps for SharePoint 420
Impacts of apps for SharePoint 420
Plan for apps for SharePoint 2013 421
Governance: determine the app for SharePoint policy for your organization 421
Plan app configuration settings 422
Determine the domain name to use 424
Plan app permissions management in SharePoint 2013 427
App permission request scopes 428
App authorization policies 430
Configure an environment for apps for SharePoint 2013 431
Configure the domain names in DNS (all hosting options) 433
Create a new wildcard SSL certificate 437
Configure the Subscription Settings and App Management service applications 437
Configure the app URLs to use 442
Configure the Internet-facing endpoints feature (Optional) 444
Manage the App Catalog in SharePoint 2013 445
Configure the App Catalog site for a web application 446
Configure app requests and SharePoint Store settings 447
Add apps to the App Catalog 449
Remove apps from the App Catalog 450
Add apps for SharePoint to a SharePoint 2013 site 451
Add apps for SharePoint to SharePoint sites 452
Remove an app for SharePoint from a SharePoint 2013 site 455
Remove an app from a SharePoint site 455
Monitor apps for SharePoint for SharePoint Server 2013 457
Selecting apps to monitor in Central Administration 458
Monitoring app details in Central Administration 459
Monitoring app details in a SharePoint site 460
Monitor and manage app licenses in SharePoint Server 2013 462
Monitoring and managing app licenses 463
Upgrade to SharePoint 2013 466
Downloadable resources about upgrade 466
TechNet articles about upgrade 466
Additional resources about upgrade 467
Get started with upgrades to SharePoint 2013 468
Downloadable resources about upgrade to SharePoint 2013 468
TechNet articles about understanding upgrade 468
Additional resources about upgrade to SharePoint 2013 470
What’s new in SharePoint 2013 upgrade 471
In-place upgrade of the farm is not supported 471
Database-attach upgrade is available for some service application databases 471
Deferred site collection upgrade 472
Site collection health checker 472
Upgrade evaluation site collections 472
Notifications for life-cycle events 473
Throttles for site collection upgrade 473
True “SharePoint 2010” instead of visual upgrade 473
Log files now in ULS format 474
Overview of the upgrade process to SharePoint 2013 475
Create the SharePoint 2013 farm 476
Copy the SharePoint 2010 Products databases 476
Upgrade SharePoint 2010 Products databases and service applications 477
Upgrade SharePoint 2010 Products site collections 479
Upgrade other SharePoint 2010 Products site collections 481
Services upgrade overview for SharePoint Server 2013 483
Database attach upgrade with services 483
Considerations for specific services 485
Upgrade farms that share services (parent and child farms) to SharePoint 2013 487
Process for upgrading farms that share services 487
Best practices for upgrading to SharePoint 2013 494
Best practices for testing upgrade 494
Best practices for upgrading to SharePoint 2013 495
Review supported editions and products for upgrading to SharePoint 2013 497
Physical topology guidance 498
Supported editions for upgrade 498
Supported cross-product upgrades 499
Plan for upgrade to SharePoint 2013 500
TechNet articles about how to plan for upgrade 500
Additional resources about how to plan for upgrade to SharePoint 2013 501
Determine strategy for upgrade to SharePoint 2013 502
How to minimize downtime during upgrade 502
Create a plan for current customizations during upgrade to SharePoint 2013 505
Identify customizations in your environment 505
Evaluate the customizations 505
Considerations for specific customizations 507
Ensure that future customizations follow best practices 510
Plan for site collection upgrades in SharePoint 2013 511
Determine the site collections that farm administrators should upgrade 511
Plan settings for upgrade notifications, self-service upgrade, and site collection creation 512
Properties that control site collection upgrade and site creation 512
Properties that control upgrade notifications 513
Plan for upgrade evaluation sites 514
Timer jobs for upgrade evaluation site collections 515
How the upgrade evaluation site collections are created 515
Plan site collection upgrade throttling and queues 515
Throttle levels for site collection upgrade 516
About site collection modes 518
Train site collection administrators 518
Plan for performance during upgrade to SharePoint 2013 520
About upgrade performance for SharePoint 2013 520
Estimate the space that you must have for the upgrade 521
Estimate how long the upgrade will take 522
Environment performance after upgrade 526
Create a communication plan for the upgrade to SharePoint 2013 527
Who is a member of the upgrade team? 527
When and what to communicate to the upgrade team 528
When and what to communicate to site users 529
Clean up an environment before an upgrade to SharePoint 2013 530
Delete unused or underused site collections and subwebs 530
Check large lists (lists with lots of data) 531
Delete excess columns from wide lists (lists with too many columns) or remove wide lists 531
Consider moving site collections into separate databases 531
Remove extraneous document versions 531
Remove unused templates, features, and Web Parts 532
Remove PowerPoint Broadcast sites 532
Finish Visual Upgrades in SharePoint 2010 Products 532
How to make structural changes 534
Test and troubleshoot an upgrade to SharePoint 2013 536
Downloadable resources about how to test and troubleshoot upgrade 536
TechNet articles about how to test and troubleshoot upgrade 537
Additional resources about how to test and troubleshoot upgrade 537
Use a trial upgrade to SharePoint 2013 to find potential issues 539
Using a virtual test environment 541
Using a physical test environment 541
Identify and install customizations 542
Copy real data to the test environment and upgrade databases 543
Review results after you upgrade databases 544
Run upgrade again, if it is necessary 544
Upgrade site collections and My Sites 544
Review results after you upgrade site collections 545
Adjust your plans and test again 545
Troubleshoot database upgrade issues in SharePoint 2013 546
General principles to identify issues 546
First, check upgrade status and log files 546
Then, address issues in order 547
Q: I want to upgrade from a pre-release version of SharePoint 2013 547
Q: The log says I have missing templates, features, or other server-side customizations 547
Q: I see errors and warnings during upgrade about connectivity or corruption 549
Q: I ran out of disk space 549
Q: I see an error about authentication 549
Q: SQL Server says I don’t have permissions 550
Q: A database will not upgrade 550
Q: I changed a database name during restore, but I cannot find the files that have that name 550
Q: I cannot back up the Search service application Administration database 550
Q: Trusted connections are not working for Excel Services after upgrade 550
Q: My workflows are no longer associated correctly 551
Troubleshoot site collection upgrade issues in SharePoint 2013 552
Check upgrade status and log files 552
Q: I don’t see a UI control on the page that used to be there 553
Q: The view on a large list is not working any longer 553
Q: I see an error about a duplicate content type name 553
Q: My site looks ugly, doesn’t behave as expected, or I see script errors 553
Q: Custom content in my site disappeared or doesn’t work 554
Q: I receive an error that says a control or page cannot render 554
Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013 555
Restart upgrade for a database by using Windows PowerShell 555
Restart upgrade for a site collection 556
Upgrade databases from SharePoint 2010 to SharePoint 2013 558
Downloadable resources about upgrading databases 558
TechNet articles about upgrading databases 558
Additional resources about upgrade 559
Checklist for database-attach upgrade (SharePoint 2013) 561
Complete the database attach upgrade 562
Prepare the new environment 562
Back up and restore databases 564
Upgrade service application databases 565
Attach and upgrade content databases 568
Complete post-upgrade steps 569
Post upgrade steps for database attach upgrade 569
Attach databases and upgrade to SharePoint 2013 572
Install SharePoint 2013 in a new environment 574
Configure service applications and farm settings 574
Record the passphrase for the Secure Store service application 576
Set the previous version databases to be read-only 576
Back up the SharePoint 2010 Products databases by using SQL Server tools 577
Export the encryption key for the User Profile service application 578
Restore a backup copy of the database 579
Set the databases to read-write 580
About upgrading the service application databases 581
Start the service instances 581
Upgrade the Secure Store service application 583
Upgrade the Business Data Connectivity service application 585
Upgrade the Managed Metadata service application 587
Upgrade the User Profile service application 588
Start the User Profile Synchronization service 591
Upgrade the PerformancePoint Services service application 592
Upgrade the Search service application 593
Verify that all of the new proxies are in the default proxy group 596
Attach a content database to a web application and upgrade the database 600
Verification: Verify upgrade for the first database 603
Attach the remaining databases 604
Verification: Verify upgrade for additional databases 604
Verify database upgrades in SharePoint 2013 606
Verify upgrade status for databases 606
Review the log files for database attach upgrade 606
Check upgrade status for databases 607
Validate the upgraded environment 607
Migrate from classic-mode to claims-based authentication in SharePoint 2013 608
Convert SharePoint 2013 classic-mode web applications to claims-based web applications 612
Upgrade site collections to SharePoint 2013 616
Downloadable resources how to upgrade site collections 616
TechNet articles about how to upgrade site collections 616
Additional resources about how to upgrade to SharePoint 2013 617
Run site collection health checks in SharePoint 2013 618
Site collection health check rules 619
Run the site collection pre-upgrade health checks by using Site Settings 620
Run the site collection pre-upgrade health checks by using Windows PowerShell 620
Upgrade a site collection to SharePoint 2013 623
Create an upgrade evaluation site (Optional) 625
View upgrade status in Site Settings 626
Review site collections upgraded to SharePoint 2013 628
Checklists for reviewing upgraded sites 629
Customized (unghosted) pages 631
Manage site collection upgrades to SharePoint 2013 633
Before you begin to upgrade site collections to SharePoint 2013 633
Control upgrade notifications and self-service upgrade 634
Control the compatibility range for site creation modes 636
Control the queue for upgrades of sites to SharePoint 2013 638
Control site throttle settings for upgrade to SharePoint 2013 640
Create upgrade evaluation site collections by using Windows PowerShell 644
Upgrade site collections by using Windows PowerShell 645
View upgrade status by using Windows PowerShell 646
- Abstract
-
Getting help
Every effort has been made to ensure the accuracy of this book. This content is also available online in the Office System TechNet Library, so if you run into problems you can check for updates at:
http://technet.microsoft.com/office
If you do not find your answer in our online content, you can send an email message to the Microsoft Office System and Servers content team at:
itspdocs@microsoft.com
If your question is about Microsoft Office products, and not about the content of this book, please search the Microsoft Help and Support Center or the Microsoft Knowledge Base at:
http://support.microsoft.com
-
Overview of SharePoint 2013 installation and configuration
Published: July 16, 2012
Summary: Learn about how to install and configure SharePoint Server 2013 or SharePoint Foundation 2013 in a farm.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Although SharePoint products farms vary in complexity and size, a combination of careful planning and a phased deployment that includes ongoing testing and evaluation significantly reduces the risk of unexpected outcomes. This article provides an overview for all types of SharePoint 2013 farm deployment.
For a visual representation of the information in this article, see the SharePoint 2013 Products Deployment model in the Technical diagrams (SharePoint 2013) topic. Related technical diagrams include “Topologies for SharePoint 2013 and Services inSharePoint Server 2013″.
In this article:
The logical result of SharePoint 2013’s flexibility and richness can be a high degree of complexity around installing and configuring SharePoint 2013 correctly. A fundamental understanding of the following key structural elements in a SharePoint 2013 environment is required in order to correctly deploy and support SharePoint 2013:
- Server farm: The top-level element of a logical architecture design for SharePoint 2013.
- Web application: An IIS Web site that is created and used by SharePoint 2013.
- Content database: Provides storage Web application content. You can separate content into multiple content databases at the site collection level.
- Site collection: A set of Web sites that have the same owner and share administration settings.
- Site: One or more related Web pages and other items (such as lists, libraries, and documents) that are hosted inside a site collection.
For more information about these and other architectural components, see Plan logical architectures for SharePoint 2013.
In addition to understanding the elements of a SharePoint 2013 environment and how they have to be configured for your solution, you must consider the following additional factors: physical architecture, installation and configuration, and the various stages of deployment.
- Server farm: The top-level element of a logical architecture design for SharePoint 2013.
-
Physical architecture
The physical architecture, which consists of one or more servers and the network infrastructure, enables you to implement the logical architecture for a SharePoint 2013 solution. The physical architecture is typically described in two ways: by its size and by its topology. Size, which can be measured in several ways, such as the number of users or the number of documents, is used to categorize a farm as small, medium, or large. Topology uses the idea of tiers or server groups to define a logical arrangement of farm servers.
-
Size
Size uses the number of users and number of content items as a fundamental measure to indicate whether a server farm is small, medium, and large, as follows:
-
A small server farm typically consists of at least two Web servers and a database server. One of the Web servers hosts the Central Administration site and the other handles additional farm-related tasks, such as serving content to users.
The small farm can be scaled out to three tiers using a dedicated application server in response to the number of users, the number of content items, and the number of services that are required.
-
A medium server farm typically consists of two or more Web servers, two application servers, and more than one database servers. We recommend that you start with the preceding configuration and then scale out to accommodate the workload placed on the servers.
In scenarios where services are known to use a disproportionate amount of resources, you can scale out the application tier. Performance data will indicate which services you should consider off-loading to a dedicated server.
- A large server farm can be the logical result of scaling out a medium farm to meet capacity and performance requirements or by design before a SharePoint 2013 solution is implemented. A three-tier topology environment typically uses dedicated servers on all the tiers. Additionally, these servers are often grouped according to their role in the farm. For example, all client-related services can be grouped onto one or two servers and then scaled out by adding servers to this group as needed in response to user demand for these services.
Note:
The recommendation for scaling out a farm is to group services or databases with similar performance characteristics onto dedicated servers and then scale out the servers as a group. In large environments, the specific groups that evolve for a farm depend on the specific demands for each service in a farm.
For specific numbers related to small, medium, and large farms, see Performance and capacity management for SharePoint 2013 Products.
Topology
Topology uses tiers as a model for logically arranging farm servers according to the components that they host or their roles in a server farm. A SharePoint 2013 farm is deployed on one, two, or three tiers, as follows:
- In a single-tier deployment, SharePoint 2013 and the database server are installed on one computer.
- In a two-tier deployment, SharePoint 2013 components and the database are installed on separate servers. This kind of deployment maps to what is called a small farm. The front-end Web servers are on the first tier and the database server is located on the second tier. In the computer industry, the first tier is known as the Web tier. The database server is known as the database tier or database back-end.
- In a three-tier deployment, the front-end Web servers are on the first tier, the application servers are on the second tier, which is known as the application tier, and the database server is located on the third tier. A three-tier deployment is used for medium and large farms.
-
-
Installation and configuration
After you finish planning your solution you can create a SharePoint 2013 farm to host the solution. The first step is to install SharePoint 2013 and create the farm that is required for the solution. The process of preparing your environment consists of the following phases:
- Prepare the servers
- Create the farm
- Configure settings, services, solutions, and sites
Note:
The farm that you create and deploy will undergo significant changes in size, topology, and complexity as you move through the different deployment stages illustrated in the SharePoint 2013 Products Deployment model. This is typical and the expected result of a phased deployment. This is why we recommend that you follow all of the stages described in the “Deployment stages” section of this article.
-
Prepare the servers
In this phase, you get your servers ready to host the product. This includes the supporting servers and the servers that will have SharePoint 2013 installed. The following servers must be configured to support and host a farm:
-
Database server: The required version of SQL Server, including service packs and cumulative updates must be installed on the database server. The installation must include any additional features, such as SQL Analysis Services, and the appropriate SharePoint 2013 logins have to be added and configured. The database server must be hardened and, if it is required, databases must be created by the DBA. For more information, see:
-
Application servers and front-end Web servers: The farm servers that will have SharePoint 2013 installed must be prepared as follows: verify that they meet the hardware requirements, have the operating system hardened, have the required networking and security protocols configured, have the SharePoint 2013 software prerequisites installed and hardened, and have the required authentication configured. For more information, see:
- Domain controller: The required farm accounts have to be configured for the domain and directory synchronization must be configured.
Important:
SharePoint 2013 does not support installation on to a domain controller in a production environment. A single label domain (SLD) names or single label forests is also not supported. Because the use of SLD names is not a recommended practice, SharePoint 2013 is not tested in this scenario. Therefore, there may be incompatibility issues when SharePoint 2013 are implemented in a single label domain environment. For more information, see Information about configuring Windows for domains with single-label DNS names and the DNS Namespace Planning Solution Center.
For information about required accounts, see:
- Initial deployment administrative and service accounts in SharePoint 2013
- About Directory Synchronization (http://go.microsoft.com/fwlink/p/?LinkId=193169)
-
Create the farm
In this phase, you install the product and configure each server to support its role in the farm. You also create the configuration database and the SharePoint Central Administration Web site. The following servers are required for a SharePoint 2013 farm:
- Database server: Unless you plan to use DBA-created databases, the configuration database, content database, and other required databases are created when you run the SharePoint Products Configuration Wizard.
- Application server: After you prepare the application server, install any additional components that are required to support functions such as Information Rights Management (IRM) and decision support. Install SharePoint 2013 on the server that will host SharePoint Central Administration Web site and then run the SharePoint Products Configuration Wizard to create and configure the farm.
- Front-end Web server: Install SharePoint 2013 on each Web server, install language packs, and then run the SharePoint Products Configuration Wizard to add the Web servers to the farm.
Note:
After you add and configure all the front-end Web servers, you can add any additional application servers that are part of your topology design to the farm.
For more information about supported deployment scenarios, see Install SharePoint 2013.
-
Configure settings, services, solutions, and sites
In this phase, you prepare the farm to host your site content by completing the following tasks:
- Configure services. For more information, see Configure services and service applications in SharePoint 2013
- Configure global settings. For more information, see Configure SharePoint 2013
- Create and populate the sites. For more information, see Set up Web applications and sites (SharePoint 2013)
Note:
Farm configuration steps are not isolated to a specific tier in the server infrastructure.
-
Deployment stages
By deploying a SharePoint 2013 solution in stages, you gain the benefits that are provided by a systematic approach, such as collecting performance and usage data that you can use to evaluate your solution. Additional benefits include verifying your capacity management assumptions and identifying issues before the farm is put into production.
We recommend that you deploy your farm in the following stages:
- Planning
- Development
- Proof of concept
- Pilot
- User acceptance test
-
Production
-
Planning
-
Before you can deploy a farm, you must plan the solution that you want to deploy and determine the infrastructure requirements, such as server resources and farm topology. When you finish the planning stage, you should have documented the following:
- An infrastructure design to support your solution
- A detailed description of how you will implement the farm and the solution
- A plan for testing and validating the solution
- A site and solution architecture
- An understanding of the monitoring and sustained engineering requirements to support the solution
- A record of how the solution will be governed
- An understanding of how the solution will be messaged to the user to drive adoption of the solution
We recommend that you use the planning resources and articles described in Plan for SharePoint 2013.
Important:
Resource and time issues may pressure you to be less rigorous during the planning stage. We recommend that you try to be as diligent as possible because missed or lightly touched planning elements can resurface as significant issues after you are in production. These issues can create much additional work, consume unbudgeted resources, and potentially take away from the success of your SharePoint 2013.
After the planning stage, you move through the following deployment stages, updating and revising your plans, configurations, and topologies as you test.
-
Development
During the development stage you will deploy SharePoint 2013 on a single server or on multiple servers to develop, test, evaluate, and refine the solution that you intend to implement. This environment is scaled according to your needs during solution development and can be retained as a scaled down environment for future development and testing. This is not a stable environment and there are no service-level agreements.
-
Proof of concept
During the proof of concept stage, the objective is two-fold: to understand SharePoint 2013 and to evaluate SharePoint 2013 in the context of how it can address your business needs. The first level of product evaluation can be done by installing all of the product components on a single server. You do a more extensive product evaluation by a proof-of-concept deployment.
A proof-of-concept deployment on a single server or on a small farm enables you to expand the scope of your evaluation. In this deployment, non-IT staff is added to the evaluation team, which provides a broader view of how SharePoint 2013 features might be actually be used in the organization. The benefit of a proof-of-concept deployment is that you can collect data that can be used to refine your original plan. This data—such as page views, user behavior patterns, and server resource consumption—also enables you to start to build a benchmark for sizing your farm. A proof of concept is also good when you evaluate service applications and determining what feature sets that you will offer your end users.
It is important during the proof-of-concept stage that you understand the unique characteristics and functionality of these features because this understanding will help you define your overall topology. Be aware that a proof-of-concept deployment requires additional resources and extends the time required to put SharePoint 2013 into production.
Tip:
Virtualization provides a good platform for evaluating SharePoint 2013 because a virtual environment provides flexibility, rapid deployment capability, and the ability to roll back virtual machines to previous states.
-
Pilot
A pilot is used to test your solution on a small scale. There are two approaches to using a pilot deployment. In the first approach, the focus is on functional testing without using real data. By using the second approach you test for production characteristics by using real data and have your pilot users test different kinds of tasks. We recommend the second approach because of the broader scope and real-world data that you can collect and use to refine your solution design.
A pilot deployment provides many benefits. It enables you to collect data that you can use to validate the following aspects of your farm design:
- Infrastructure design
- Capacity management assumptions
- Site and solution architecture
- Solution usage assumptions
The pilot stage also enables you to determine additional data that should be collected to increase the breadth and depth of your benchmarks. This is important if you want to assess the potential effect of additional features or services that you want to add to the farm before the user acceptance test.
At the conclusion of the pilot deployment, you can use the data that you collect to adjust the various components of the solution and its supporting infrastructure.
-
User acceptance test (UAT)
A user acceptance test deployment—also known as a pre-production environment—is used by organizations as a transitional step from the pilot deployment to a production deployment. An organization’s business processes determine the scope, scale, and duration of user accept testing.
The topology of the pre-production environment should be the same as, or very similar to the planned production topology. During user acceptance testing, the SharePoint 2013 solution is tested against a subset or a complete copy of production data. This deployment stage provides a final opportunity for performance tuning and validating operational procedures such as backups and restores.
-
Production
The final stage is rolling your farm into a production environment. At this stage, you will have incorporated the necessary solution and infrastructure adjustments that were identified during the user acceptance test stage.
Putting the farm into production requires you to complete the following tasks:
- Deploy the farm.
- Deploy the solution.
- Implement the operations plan.
- If required, deploy additional environments such as authoring and staging farms, and services farms.
- Planning
-
Prepare for installation of SharePoint 2013
Published: July 16, 2012
Summary: Learn about permissions, accounts, security settings, and what you have to do to prepare your environment for SharePoint 2013.
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about how to prepare for SharePoint 2013 installation and initial configuration.
-
TechNet articles about how to prepare for SharePoint 2013 installation and initial configuration
The following articles about how to prepare for SharePoint 2013 installation and initial configuration are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Initial deployment administrative and service accounts in SharePoint 2013
Provides information about the administrative and service accounts that are required for an initial SharePoint 2013 installation.
Account permissions and security settings in SharePoint 2013
Describes SharePoint 2013 administrative and services account permissions. This article discusses the following areas: Microsoft SQL Server, the file system, file shares, and registry entries.
Configure SQL Server security for SharePoint 2013 environments
Learn how to harden SQL Server for SharePoint 2013 environments by using SQL Server tools and Windows Firewall.
Install prerequisites for SharePoint 2013 from a network share
Describes how to install SharePoint 2013 prerequisites from an offline shared network location using the prerequisite installer (PrerequisiteInstaller.exe) tool.
-
Additional resources about SharePoint 2013 installation and initial configuration
The following resources about SharePoint 2013 installation and initial configuration are available from other subject matter experts.
Content
Description
Visit the Resource Center to access videos, Community Sites, documentation, and more.
-
Initial deployment administrative and service accounts in SharePoint 2013
Updated: October 2, 2012
Summary: Learn about the administrative and service accounts that are required to initially install SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
This article provides information about the administrative and service accounts that are required for an initial SharePoint 2013 deployment. Additional accounts and permissions are required to fully implement all aspects of a production farm.
Note:
For a complete list of permissions, see Account permissions and security settings in SharePoint 2013.
-
Required permissions
To deploy SharePoint 2013 on a server farm, you must provide credentials for several different accounts.
The following table describes the accounts that are used to install and configure SharePoint 2013.
Account
Purpose
Requirements
SQL Server service account
The SQL Server service account is used to run SQL Server. It is the service account for the following SQL Server services:
- MSSQLSERVER
- SQLSERVERAGENT
If you do not use the default SQL Server instance, in the Windows Services console, these services will be shown as the following:
- MSSQL<InstanceName>
- SQLAgent<InstanceName>
Use either a Local System account or a domain user account.
If you plan to back up to or restore from an external resource, permissions to the external resource must be granted to the appropriate account. If you use a domain user account for the SQL Server service account, grant permissions to that domain user account. However, if you use the Network Service or the Local System account, grant permissions to the external resource to the machine account (<domain_name>\<SQL_hostname>).
The instance name is arbitrary and was created when SQL Server was installed.
Setup user account
The Setup user account is used to run the following:
- Setup
- SharePoint Products Configuration Wizard
- Domain user account.
- Member of the Administrators group on each server on which Setup is run.
- SQL Server login on the computer that runs SQL Server.
-
Member of the following SQL Server roles:
- securityadmin fixed server role
- dbcreator fixed server role
- securityadmin fixed server role
If you run Windows PowerShell cmdlets that affect a database, this account must be a member of the db_owner fixed database role for the database.
Server farm account or database access account
The server farm account is used to perform the following tasks:
- Configure and manage the server farm.
- Act as the application pool identity for the SharePoint Central Administration Web site.
- Run the Microsoft SharePoint Foundation Workflow Timer Service.
- Domain user account.
Additional permissions are automatically granted for the server farm account on Web servers and application servers that are joined to a server farm.
The server farm account is automatically added as a SQL Server login on the computer that runs SQL Server. The account is added to the following SQL Server security roles:
- dbcreator fixed server role
- securityadmin fixed server role
- db_owner fixed database role for all SharePoint databases in the server farm
Note:
We recommend that you install SharePoint 2013 by using least-privilege administration.
- MSSQLSERVER
-
Account permissions and security settings in SharePoint 2013
Published: September 4, 2012
Summary: Learn about the permissions and security settings to use with a deployment of SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article describes SharePoint administrative and services account permissions for the following areas: Microsoft SQL Server, the file system, file shares, and registry entries.
In this article:
-
About account permissions and security settings
The SharePoint Configuration Wizard (Psconfig) and the Farm Creation Wizard, both of which are run during a Complete installation, configure many of the SharePoint baseline account permissions and security settings.
-
SharePoint administrative accounts
One of the following SharePoint components automatically configures most of the SharePoint administrative account permissions during the setup process:
- The SharePoint Configuration Wizard (Psconfig).
- The Farm Creation Wizard.
- The SharePoint Central Administration web site.
-
Windows PowerShell.
-
Setup user administrator account
-
This account is used to set up each server in your farm by running the SharePoint Configuration Wizard, the initial Farm Creation Wizard, and Windows PowerShell. For the examples in this article, the setup user administrator account is used for farm administration, and you can use Central Administration to manage it. Some configuration options, for example, configuration of the SharePoint 2013 Search query server, require local administration permissions. The setup user administrator account requires the following permissions:
- It must have domain user account permissions.
- It must be a member of the local administrators group on each server in the SharePoint farm, excluding the server running SQL Server and the Simple Mail Transfer Protocol (SMTP) server.
- This account must have access to the SharePoint databases.
- If you use any Windows PowerShell operations that affect a database, the setup user administrator account must be a member of the db_owner role.
- This account must be assigned to the securityadmin and dbcreatorSQL Server security roles during setup and configuration.
Note:
The securityadmin and dbcreatorSQL Server security roles might be required for this account during a complete version-to-version upgrade because new databases might have to be created and secured for services.
After you run the configuration wizards, machine-level permissions for the setup user administrator account include:
- Membership in the WSS_ADMIN_WPG Windows security group.
- Membership in the IIS_WPG role.
After you run the configuration wizards, database permissions include:
- db_owner on the SharePoint server farm configuration database.
- db_owner on the SharePoint Central Administration content database.
Warning:
If the setup user administrator account cannot a log on to the computer running SQL Server, the configuration wizards will not run correctly. If the account that you use to run the configuration wizards does not have the appropriate special SQL Server role membership or access as db_owner on the databases, the configuration wizards will not run correctly.
-
SharePoint farm service account
The server farm account, which is also referred to as the database access account, is used as the application pool identity for Central Administration and as the process account for the SharePoint Foundation 2013 Timer service. The server farm account requires the following permissions:
- It must have domain user account permissions.
Additional permissions are automatically granted to the server farm account on web servers and application servers that are joined to a server farm.
After you run the SharePoint Configuration Wizard, machine-level permissions include:
- Membership in the WSS_ADMIN_WPG Windows security group for the SharePoint Foundation 2013 Timer service.
- Membership in WSS_RESTRICTED_WPG for the Central Administration and Timer service application pools.
- Membership in WSS_WPG for the Central Administration application pool.
After you run the configuration wizards, SQL Server and database permissions include:
- Dbcreator fixed server role.
- Securityadmin fixed server role.
- db_owner for all SharePoint databases.
- Membership in the WSS_CONTENT_APPLICATION_POOLS role for the SharePoint server farm configuration database.
- Membership in the WSS_CONTENT_APPLICATION_POOLS role for the SharePoint_Admin content database.
- The SharePoint Configuration Wizard (Psconfig).
-
SharePoint service application accounts
This section describes the service application accounts that are set up by default during installation.
-
Application pool account
The application pool account is used for application pool identity. The application pool account requires the following permission configuration settings:
The following machine-level permission is configured automatically: The application pool account is a member of WSS_WPG.
The following SQL Server and database permissions for this account are configured automatically:
- The application pool accounts for Web applications are assigned to the SP_DATA_ACCESS role for the content databases.
- This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the farm configuration database.
-
This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the SharePoint_Admin content database.
-
Default content access account
-
Important:
Information in this section applies to SharePoint Server 2013 only.
The default content access account is used within a specific service application to crawl content, unless a different authentication method is specified by a crawl rule for a URL or URL pattern. This account requires the following permission configuration settings:
- The default content access account must be a domain user account that has read access to external or secure content sources that you want to crawl by using this account.
- For SharePoint Server sites that are not part of the server farm, you have to explicitly grant this account full read permissions to the web applications that host the sites.
-
This account must not be a member of the Farm Administrators group.
-
Content access accounts
-
Important:
Information in this section applies to SharePoint Server 2013 only.
Content access accounts are configured to access content by using the Search administration crawl rules feature. This type of account is optional and you can configure it when you create a new crawl rule. For example, external content (such as a file share) might require this separate content access account. This account requires the following permission configuration settings:
- The content access account must have read access to external or secure content sources that this account is configured to access.
-
For SharePoint Server sites that are not part of the server farm, you have to explicitly grant this account full read permissions to the web applications that host the sites.
-
Excel Services unattended service account
-
Important:
Information in this section applies to SharePoint Server 2013 only.
Excel Services uses the Excel Services unattended service account to connect to external data sources that require a user name and password that are based on operating systems other than Windows for authentication. If this account is not configured, Excel Services will not attempt to connect to these types of data sources. Although account credentials are used to connect to data sources of operating systems other than Windows, if the account is not a member of the domain, Excel Services cannot access them. This account must be a domain user account.
-
My Sites application pool account
Important:
Information in this section applies to SharePoint Server 2013 only.
The My Sites application pool account must be a domain user account. This account must not be a member of the Farm Administrators group.
The following machine-level permission is configured automatically: This account is a member of WSS_WPG.
The following SQL Server and database permissions are configured automatically:
- This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role that is associated with the farm configuration database.
- This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role that is associated with the SharePoint_Admin content database.
-
The application pool accounts for web applications are assigned to the SP_DATA_ACCESS role for the content databases
-
Other application pool accounts
-
The other application pool account must be a domain user account. This account must not be a member of the Administrators group on any computer in the server farm.
The following machine-level permission is configured automatically: This account is a member of WSS_WPG.
The following SQL Server and database permissions are configured automatically:
- This account is assigned to the SP_DATA_ACCESS role for the content databases.
- This account is assigned to the SP_DATA_ACCESS role for search database that is associated with the web application.
- This account must have read and write access to the associated service application database.
- This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role that is associated with the farm configuration database.
- This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role that is associated with the SharePoint_Admin content database.
-
-
SharePoint database roles
This section describes the database roles that installation sets up by default or that you can configure optionally.
-
WSS_CONTENT_APPLICATION_POOLS database role
The WSS_CONTENT_APPLICATION_POOLS database role applies to the application pool account for each web application that is registered in a SharePoint farm. This enables web applications to query and update the site map and have read-only access to other items in the configuration database. Setup assigns the WSS_CONTENT_APPLICATION_POOLS role to the following databases:
- The SharePoint_Config database (the configuration database).
- The SharePoint_AdminContent database.
Members of the WSS_CONTENT_APPLICATION_POOLS role have the execute permission for a subset of the stored procedures for the database. In addition, members of this role have the select permission to the Versions table (dbo.Versions) in the SharePoint_AdminContent database. For other databases, the accounts planning tool indicates that access to read these databases is automatically configured. In some cases, limited access to write to a database is also automatically configured. To provide this access, permissions for stored procedures are configured.
-
WSS_SHELL_ACCESS database role
The secure WSS_SHELL_ACCESS database role on the configuration database replaces the need to add an administration account as a db_owner on the configuration database. By default, the setup account is assigned to the WSS_SHELL_ACCESS database role. You can use a Windows PowerShell command to grant or remove memberships to this role. Setup assigns the WSS_SHELL_ACCESS role to the following databases:
- The SharePoint_Config database (the configuration database).
- One or more of the SharePoint Content databases. This is configurable by using the Windows PowerShell command that manages membership and the object that is assigned to this role.
Members of the WSS_SHELL_ACCESS role have the execute permission for all stored procedures for the database. In addition, members of this role have the read and write permissions on all of the database tables.
-
SP_READ_ONLY database role
The SP_READ_ONLY role should be used for setting the database to read only mode instead of using sp_dboption. This role as its name suggests should be used when only read access is required for data such as usage and telemetry data.
Note:
The sp_dboption stored procedure is not available in SQL Server 2012. For more information about sp_dboption see sp_dboption.
The SP_READ_ONLY SQL role will have the following permissions:
- Grant SELECT on all SharePoint stored procedures and functions
- Grant SELECT on all SharePoint tables
-
Grant EXECUTE on user-defined type where schema is dbo
-
SP_DATA_ACCESS database role
-
The SP_DATA_ACCESS role is the default role for database access and should be used for all object model level access to databases. Add the application pool account to this role during upgrade or new deployments.
Note:
The SP_DATA_ACCESS role replaces the db_owner role in SharePoint 2013.
The SP_DATA_ACCESS role will have the following permissions:
- Grant EXECUTE or SELECT on all SharePoint stored procedures and functions
- Grant SELECT on all SharePoint tables
- Grant EXECUTE on User-defined type where schema is dbo
- Grant INSERT on AllUserDataJunctions table
- Grant UPDATE on Sites view
- Grant UPDATE on UserData view
- Grant UPDATE on AllUserData table
- Grant INSERT and DELETE on NameValuePair tables
- Grant create table permission
-
-
Group permissions
This section describes permissions of groups that the SharePoint 2013 setup and configuration tools create.
-
WSS_ADMIN_WPG
WSS_ADMIN_WPG has read and write access to local resources. The application pool accounts for the Central Administration and Timer services are in WSS_ADMIN_WPG. The following table shows the WSS_ADMIN_WPG registry entry permissions.
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS
Full control
Not Applicable
Not Applicable
HKEY_LOCAL_MACHINE\Software\Microsoft\Office\15.0\Registration\{90150000-110D-0000-1000-0000000FF1CE}
Read, write
Not Applicable
Not Applicable
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server
Read
No
This key is the root of the SharePoint 2013 registry settings tree. If this key is altered, SharePoint 2013 functionality will fail.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0
Full control
No
This key is the root of the SharePoint 2013 registry settings.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LoadBalancerSettings
Read, write
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LauncherSettings
Read, write
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\Search
Full control
Not Applicable
Not Applicable
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Search
Full control
Not Applicable
Not Applicable
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure
Full control
No
This key contains the connection string and the ID of the configuration database to which the machine is joined. If this key is altered, the SharePoint 2013 installation on the machine will not function.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS
Full control
Yes
This key contains settings used during setup. If this key is altered, diagnostic logging may fail and setup or post-setup configuration may fail.
The following table shows the WSS_ADMIN_WPG file system permissions.
File system path
Permissions
Inherit
Description
%AllUsersProfile%\ Microsoft\SharePoint
Full control
No
This directory contains the file-system-backed cache of the farm configuration. Processes might fail to start and the administrative actions might fail if this directory is altered or deleted.
C:\Inetpub\wwwroot\wss
Full control
No
This directory (or the corresponding directory under the Inetpub root on the server) is used as the default location for IIS Web sites. SharePoint sites will be unavailable and administrative actions might fail if this directory is altered or deleted, unless custom IIS Web site paths are provided for all IIS Web sites extended with SharePoint 2013.
%ProgramFiles%\Microsoft Office Servers\15.0
Full control
No
This directory is the installation location for SharePoint 2013 binaries and data. The directory can be changed during installation. All SharePoint 2013 functionality will fail if this directory is removed, altered, or removed after installation. Membership in the WSS_ADMIN_WPG Windows security group is required for some SharePoint 2013 services to be able to store data on disk.
%ProgramFiles%\Microsoft Office Servers\15.0\WebServices
Read, write
No
This directory is the root directory where back-end Web services are hosted, for example, Excel and Search. The SharePoint 2013 features that depend on these services will fail if this directory is removed or altered.
%ProgramFiles%\Microsoft Office Servers\15.0\Data
Full control
No
This directory is the root location where local data is stored, including search indexes. Search functionality will fail if this directory is removed or altered. WSS_ADMIN_WPG Windows security group permissions are required to enable search to save and secure data in this folder.
%ProgramFiles%\Microsoft Office Servers\15.0\Logs
Full control
Yes
This directory is the location where the run-time diagnostic logging is generated. Logging functionality will not function properly if this directory is removed or altered.
%ProgramFiles%\Microsoft Office Servers\15.0\Data\Office Server
Full control
Yes
Same as the parent folder.
%windir%\System32\drivers\etc\HOSTS
Read, write
Not Applicable
Not Applicable
%windir%\Tasks
Full control
Not Applicable
Not Applicable
%COMMONPROGRAMFILES%Microsoft Shared\Web Server Extensions\15
Modify
Yes
This directory is the installation directory for core SharePoint 2013 files. If the access control list (ACL) is modified, feature activation, solution deployment, and other features will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\ADMISAPI
Full control
Yes
This directory contains the SOAP services for Central Administration. If this directory is altered, remote site creation and other methods exposed in the service will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\CONFIG
Full control
Yes
This directory contains files used to extend IIS Web sites with SharePoint 2013. If this directory or its contents are altered, web application provisioning will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS
Full control
No
This directory contains setup and runtime tracing logs. If the directory is altered, diagnostic logging will not function correctly.
%windir%\temp
Full control
Yes
This directory is used by platform components on which SharePoint 2013 depends. If the access control list is modified, Web Part rendering and other deserialization operations might fail.
%windir%\System32\logfiles\SharePoint
Full control
No
This directory is used by SharePoint Server usage logging. If this directory is modified, usage logging will not function correctly.
This registry key applies only to SharePoint Server.
%systemdrive\program files\Microsoft Office Servers\15 folder on Index servers
Full control
Not Applicable
This permission is granted for a %systemdrive\program files\Microsoft Office Servers\15 folder on Index servers.
-
WSS_WPG
WSS_WPG has read access to local resources. All application pool and services accounts are in WSS_WPG. The following table shows WSS_WPG registry entry permissions.
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0
Read
No
This key is the root of the SharePoint 2013 registry settings.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\Diagnostics
Read, write
No
This key contains settings for the SharePoint 2013 diagnostic logging. Altering this key will break the logging functionality.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LoadBalancerSettings
Read, write
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LauncherSettings
Read, write
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure
Read
No
This key contains the connection string and the ID of the configuration database to which the machine is joined. If this key is altered, the SharePoint 2013 installation on the machine will not function.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS
Read
Yes
This key contains settings that are used during setup. If this key is altered, diagnostic logging may fail and setup or post-setup configuration may fail.
The following table shows the WSS_WPG file system permissions.
File system path
Permissions
Inherit
Description
%AllUsersProfile%\ Microsoft\SharePoint
Read
No
This directory contains the file-system-backed cache of the farm configuration. Processes might fail to start and the administrative actions might fail if this directory is altered or deleted.
C:\Inetpub\wwwroot\wss
Read, execute
No
This directory (or the corresponding directory under the Inetpub root on the server) is used as the default location for IIS Web sites. SharePoint sites will be unavailable and administrative actions might fail if this directory is altered or deleted, unless custom IIS Web site paths are provided for all IIS Web sites extended with SharePoint 2013.
%ProgramFiles%\Microsoft Office Servers\15.0
Read, execute
No
This directory is the installation location for the SharePoint 2013 binaries and data. It can be changed during installation. All SharePoint 2013 functionality will fail if this directory is removed, altered, or moved after installation. WSS_WPG read and execute permissions are required to enable IIS sites to load SharePoint 2013 binaries.
%ProgramFiles%\Microsoft Office Servers\15.0\WebServices
Read
No
This directory is the root directory where back-end Web services are hosted, for example, Excel and Search. The SharePoint 2013 features that depend on these services will fail if this directory is removed or altered.
%ProgramFiles%\Microsoft Office Servers\15.0\Logs
Read, write
Yes
This directory is the location where the runtime diagnostic logging is generated. Logging functionality will not function properly if this directory is removed or altered.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\ADMISAPI
Read
Yes
This directory contains the SOAP services for Central Administration. If this directory is altered, remote site creation and other methods exposed in the service will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\CONFIG
Read
Yes
This directory contains files used to extend IIS Web sites with SharePoint 2013. If this directory or its contents are altered, web application provisioning will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS
Modify
No
This directory contains setup and runtime tracing logs. If the directory is altered, diagnostic logging will not function correctly.
%windir%\temp
Read
Yes
This directory is used by platform components on which SharePoint 2013 depends. If the access control list is modified, Web Part rendering, and other deserialization operations may fail.
%windir%\System32\logfiles\SharePoint
Read
No
This directory is used by SharePoint Server usage logging. If this directory is modified, usage logging will not function correctly.
The registry key applies only to SharePoint Server.
%systemdrive\program files\Microsoft Office Servers\15
Read, execute
Not Applicable
The permission is granted for %systemdrive\program files\Microsoft Office Servers\15 folder on Index servers.
-
Local service
The following table shows the local service registry entry permission:
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LoadBalancerSettings
Read
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
The following table shows the local service file system permission:
File system path
Permissions
Inherit
Description
%ProgramFiles%\Microsoft Office Servers\15.0\Bin
Read, execute
No
This directory is the installed location of the SharePoint 2013 binaries. All the SharePoint 2013 functionality will fail if this directory is removed or altered.
-
Local system
The following table shows the local system registry entry permissions:
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\LauncherSettings
Read
No
This key contains settings for the document conversion service. Altering this key will break document conversion functionality.
This registry key applies only to SharePoint Server.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure
Full control
No
This key contains the connection string and the ID of the configuration database to which the machine is joined. If this key is altered, the SharePoint 2013 installation on the machine will not function.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure\FarmAdmin
Full control
No
This key contains the encryption key that is used to store secrets in the configuration database. If this key is altered, service provisioning and other features will fail.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS
Full control
Yes
This key contains settings that are used during setup. If this key is altered, diagnostic logging may fail and setup or post-setup configuration may fail.
The following table shows the local file system permissions:
File system path
Permissions
Inherit
Description
%AllUsersProfile%\ Microsoft\SharePoint
Full control
No
This directory contains the file-system-backed cache of the farm configuration. Processes might fail to start and administrative actions might fail if this directory is altered or deleted.
C:\Inetpub\wwwroot\wss
Full control
No
This directory (or the corresponding directory under the Inetpub root on the server) is used as the default location for IIS Web sites. SharePoint sites will be unavailable and administrative actions might fail if this directory is altered or deleted, unless custom IIS Web site paths are provided for all IIS Web sites extended with SharePoint 2013.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\ADMISAPI
Full control
Yes
This directory contains the SOAP services for Central Administration. If this directory is altered, remote site creation and other methods exposed in the service will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\CONFIG
Full control
Yes
If this directory or its contents are altered, Web Application provisioning will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS
Full control
No
This directory contains setup and run-time tracing logs. If the directory is altered, diagnostic logging will not function correctly.
%windir%\temp
Full control
Yes
This directory is used by platform components on which SharePoint 2013 depends. If the access control list is modified, Web Part rendering, and other deserialization operations might fail.
%windir%\System32\logfiles\SharePoint
Full control
No
This directory is used by SharePoint Server for usage logging. If this directory is modified, usage logging will not function correctly.
This registry key applies only to SharePoint Server.
-
Network service
The following table shows the network service registry entry permission:
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\Software\Microsoft\Office Server\15.0\Search\Setup
Read
Not Applicable
Not Applicable
-
Administrators
The following table shows the administrators registry entry permissions:
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure
Full control
No
This key contains the connection string and the ID of the configuration database to which the machine is joined. If this key is altered, the SharePoint 2013 installation on the machine will not function.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure\FarmAdmin
Full control
No
This key contains the encryption key that is used to store secrets in the configuration database. If this key is altered, service provisioning and other features will fail.
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS
Full control
Yes
This key contains settings that are used during setup. If this key is altered, diagnostic logging may fail and setup or post-setup configuration may fail.
The following table shows the administrators file system permissions:
File system path
Permissions
Inherit
Description
%AllUsersProfile%\ Microsoft\SharePoint
Full control
No
This directory contains the file-system-backed cache of the farm configuration. Processes might fail to start and administrative actions might fail if this directory is altered or deleted.
C:\Inetpub\wwwroot\wss
Full Control
No
This directory (or the corresponding directory under the Inetpub root on the server) is used as the default location for IIS Web sites. SharePoint sites will be unavailable and administrative actions might fail if this directory is altered or deleted, unless custom IIS web site paths are provided for all IIS web sites that are extended with SharePoint 2013.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\ADMISAPI
Full control
Yes
This directory contains the SOAP services for Central Administration. If this directory is altered, remote site creation and other methods exposed in the service will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\CONFIG
Full control
Yes
If this directory or its contents are altered, web application provisioning will not function correctly.
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS
Full control
No
This directory contains setup and runtime tracing logs. If the directory is altered, diagnostic logging will not function correctly.
%windir%\temp
Full control
Yes
This directory is used by platform components on which SharePoint 2013 depends. If the ACL is modified, Web Part rendering, and other deserialization operations might fail.
%windir%\System32\logfiles\SharePoint
Full control
No
This directory is used by SharePoint Server for usage logging. If this directory is modified, usage logging will not function correctly.
This registry key applies only to SharePoint Server.
-
WSS_RESTRICTED_WPG
WSS_RESTRICTED_WPG can read the encrypted farm administration credential registry entry. WSS_RESTRICTED_WPG is only used for encryption and decryption of passwords that are stored in the configuration database. The following table shows the WSS_RESTRICTED_WPG registry entry permission:
Key name
Permissions
Inherit
Description
HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure\FarmAdmin
Full control
No
This key contains the encryption key that is used to store secrets in the configuration database. If this key is altered, service provisioning and other features will fail.
-
Users group
The following table shows the users group file system permissions:
File system path
Permissions
Inherit
Description
%ProgramFiles%\Microsoft Office Servers\15.0
Read, execute
No
This directory is the installation location for SharePoint 2013 binaries and data. It can be changed during installation. All SharePoint 2013 functionality will fail if this directory is removed, altered, or moved after installation.
%ProgramFiles%\Microsoft Office Servers\15.0\WebServices\Root
Read, execute
No
This directory is the root directory where back-end root Web services are hosted. The only service initially installed on this directory is a search global administration service. Some search administration functionality that uses the server-specific Central Administration Settings page will not work if this directory is removed or altered.
%ProgramFiles%\Microsoft Office Servers\15.0\Logs
Read, write
Yes
This directory is the location where the run-time diagnostic logging is generated. Logging will not function properly if this directory is removed or altered.
%ProgramFiles%\Microsoft Office Servers\15.0\Bin
Read, execute
No
This directory is the installed location of SharePoint 2013 binaries. All of the SharePoint 2013 functionality will fail if this directory is removed or altered.
-
All SharePoint 2013 service accounts
The following table shows the all SharePoint 2013 service accounts file system permission:
File system path
Permissions
Inherit
Description
%COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS
Modify
No
This directory contains setup and runtime tracing logs. If this directory is altered, diagnostic logging will not function correctly. All SharePoint 2013 service accounts must have write permission to this directory.
-
-
Configure SQL Server security for SharePoint 2013 environments
Published: July 16, 2012
Summary: Learn how to improve the security of SQL Server for SharePoint 2013 environments.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
When you install SQL Server, the default settings help to provide a safe database. In addition, you can use SQL Server tools and Windows Firewall to add additional security to SQL Server for SharePoint 2013 environments.
In this article:
-
Before you begin
Before you begin this operation, review the following tasks about how to secure your server farm:
- Block UDP port 1434.
- Configure named instances of SQL Server to listen on a nonstandard port (other than TCP port 1433 or UDP port 1434).
- For additional security, block TCP port 1433 and reassign the port that is used by the default instance to a different port.
- Configure SQL Server client aliases on all front-end web servers and application servers in the server farm. After you block TCP port 1433 or UDP port 1434, SQL Server client aliases are necessary on all computers that communicate with the computer that is running SQL Server.
Note:
Because SharePoint 2013 runs as websites in IIS, administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Block UDP port 1434.
-
Configuring a SQL Server instance to listen on a non-default port
SQL Server provides the ability to reassign the ports that are used by the default instance and any named instances. In SQL Server 2008 R2, and SQL Server 2012, you reassign the TCP port by using SQL Server Configuration Manager. When you change the default ports, you make the environment more secure against hackers who know default assignments and use them to exploit your SharePoint environment.
To configure a SQL Server instance to listen on a non-default port
- Verify that the user account that is performing this procedure is a member of either the sysadmin or the serveradmin fixed server role.
- On the computer that is running SQL Server, open SQL Server Configuration Manager.
- In the navigation pane, expand SQL Server Network Configuration.
-
Click the corresponding entry for the instance that you are configuring.
The default instance is listed as Protocols for MSSQLSERVER. Named instances will appear as Protocols for named_instance.
- In the main window in the Protocol Name column, right-click TCP/IP, and then click Properties.
-
Click the IP Addresses tab.
For every IP address that is assigned to the computer that is running SQL Server, there is a corresponding entry on this tab. By default, SQL Server listens on all IP addresses that are assigned to the computer.
- To globally change the port that the default instance is listening on, follow these steps:
- For each IP address except IPAll, clear all values for both TCP dynamic ports and TCP Port.
- For IPAll, clear the value for TCP dynamic ports. In the TCP Port field, enter the port that you want the instance of SQL Server to listen on. For example, enter 40000.
- To globally change the port that a named instance is listening on, follow these steps:
- For each IP address including IPAll, clear all values for TCP dynamic ports. A value of 0 for this field indicates that SQL Server uses a dynamic TCP port for the IP address. A blank entry for this value means that SQL Server will not use a dynamic TCP port for the IP address.
- For each IP address except IPAll, clear all values for TCP Port.
- For IPAll, clear the value for TCP dynamic ports. In the TCP Port field, enter the port that you want the instance of SQL Server to listen on. For example, enter 40000.
-
Click OK.
A message indicates that that the change will not take effect until the SQL Server service is restarted. Click OK.
- Close SQL Server Configuration Manager.
-
Restart the SQL Server service and confirm that the computer that is running SQL Server is listening on the port that you selected.
You can confirm this by looking in the Event Viewer log after you restart the SQL Server service. Look for an information event similar to the following event:
Event Type:Information
Event Source:MSSQL$MSSQLSERVER
Event Category:(2)
Event ID:26022
Date:3/6/2008
Time:1:46:11 PM
User:N/A
Computer:computer_name
Description:
Server is listening on [ ‘any’ <ipv4>50000]
- Verification: Optionally, include steps that users should perform to verify that the operation was successful.
-
Blocking default SQL Server listening ports
Windows Firewall with Advanced Security uses Inbound Rules and Outbound Rules to help secure incoming and outgoing network traffic. Because Windows Firewall blocks all incoming unsolicited network traffic by default, you do not have to explicitly block the default SQL Server listening ports. For more information, see Windows Firewall with Advanced Security and Configuring the Windows Firewall to Allow SQL Server Access.
-
Configuring Windows Firewall to open manually assigned ports
To access a SQL Server instance through a firewall, you must configure the firewall on the computer that is running SQL Server to allow access. Any ports that you manually assign must be open in Windows Firewall.
To configure Windows Firewall to open manually assigned ports
- Verify that the user account that is performing this procedure is a member of either the sysadmin or the serveradmin fixed server role.
- In Control Panel, open System and Security.
- Click Windows Firewall, and then click Advanced Settings to open the Windows Firewall with Advanced Security dialog box.
- In the navigation pane, click Inbound Rules to display the available options in the Actions pane.
- Click New Rule to open the New Inbound Rule Wizard.
- Use the wizard to complete the steps that are required to allow access to the port that you defined in Configuring a SQL Server instance to listen on a non-default port.
Note:
You can configure the Internet Protocol security (IPsec) to help secure communication to and from your computer that is running SQL Server by configuring the Windows firewall. You do this by selecting Connection Security Rules in the navigation pane of the Windows Firewall with Advanced Security dialog box.
-
Configuring SQL Server client aliases
If you block UDP port 1434 or TCP port 1433 on the computer that is running SQL Server, you must create a SQL Server client alias on all other computers in the server farm. You can use SQL Server client components to create a SQL Server client alias for computers that connect to SQL Server.
- Verify that the user account that is performing this procedure is a member of either the sysadmin or the serveradmin fixed server role.
- Run Setup for SQL Server on the target computer, and install the following client components:
- Connectivity Components
- Management Tools
- Open SQL Server Configuration Manager.
- In the navigation pane, click SQL Native Client Configuration.
- In the main window under Items, right-click Aliases, and select New Alias.
- In the Alias – New dialog box, in the Alias Name field, enter a name for the alias. For example, enter SharePoint_alias.
- In the Port No field, enter the port number for the database instance. For example, enter 40000. Make sure that the protocol is set to TCP/IP.
- In the Server field, enter the name of the computer that is running SQL Server.
- Click Apply, and then click OK.
- Verification: You can test the SQL Server client alias by using SQL Server Management Studio, which is available when you install SQL Server client components.
- Open SQL ServerManagement Studio.
- When you are prompted to enter a server name, enter the name of the alias that you created, and then click Connect. If the connection is successful, SQL ServerManagement Studio is populated with objects that correspond to the remote database.
- To check connectivity to additional database instances from SQL ServerManagement Studio, click Connect, and then click Database Engine.
-
Install prerequisites for SharePoint 2013 from a network share
Published: July 16, 2012
Summary: Learn how to how to install SharePoint 2013 prerequisites from an offline shared network location by using the prerequisite installer (PrerequisiteInstaller.exe) tool.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Installing prerequisites from an offline location is typically required when the servers on which you want to install SharePoint 2013 are isolated from the Internet. Even if this is not the case, installing prerequisites from an offline central location enables you to make sure of farm server consistency by installing a well-known and controlled set of images.
Note:
The Microsoft SharePoint Products Preparation Tool is a user interface built on PrerequisiteInstaller.exe. The Microsoft SharePoint Products Preparation Tool accepts no user input.
In this article:
- Installer switches and arguments
- Download and combine the SharePoint 2013 prerequisites on a file share
- Install the SharePoint 2013 prerequisites at the command prompt
- Install the SharePoint 2013 prerequisites by using an arguments file
- Known issues
Important:
The steps in this article apply to SharePoint Foundation 2013 and SharePoint Server 2013.
- Installer switches and arguments
-
Installer switches and arguments
By using PrerequisiteInstaller.exe with switches and arguments, you control the versions of the required software that are installed and the location from which they are installed.
PrequisiteInstaller.exe accepts single or multiple switch and argument pairs. A switch identifies the prerequisite and the argument specifies the action and the location of the prerequisite.
A switch and argument pair uses the following format:
/switch: <path>
Where:
- /switch is a valid switch to identify a prerequisite. For example, /SQLNCli: is the switch for the Microsoft SQL Server 2008 R2 SP1 Native Client.
- <path> is expressed as the path of a local file or the path of a file share, for example, “C:\foldername\sqlncli.msi” or “\\<servername>\<sharename>\sqlncli.msi”.
Each switch and its argument are separated by a colon and a space. The argument is enclosed in quotation marks.
The switch and argument pairs can be passed to PrerequisiteInstaller.exe at the command prompt or read from an arguments text file.
- /switch is a valid switch to identify a prerequisite. For example, /SQLNCli: is the switch for the Microsoft SQL Server 2008 R2 SP1 Native Client.
-
Download and combine the SharePoint 2013 prerequisites on a file share
The process for downloading and combining prerequisites consists of the steps that were described in the following procedures.
To identify prerequisites
- Refer to Hardware and software requirements (SharePoint 2013), which lists all the required and optional software for SharePoint 2013. Additionally, this document provides the download location for each prerequisite that is available for download on the Internet.
- From the command prompt, navigate to the root of the SharePoint 2013 installation media or folder location.
-
At the command prompt, type the following command and then press ENTER:
PrerequisiteInstaller.exe /?
This displays a list of the command-line options and switches and their corresponding arguments for installing a prerequisite from the command-line.
Tip:
To copy the contents of the active About window to the Clipboard, press CTRL+C.
- Verify that you have an accurate list of the required software. Compare the output from the prerequisite installer to the list of prerequisites in step 1.
- Download the prerequisites to a computer that has Internet access.
Next, follow these steps to create a central location that you can use for installing SharePoint 2013 prerequisites on all the farm servers.
To combine prerequisites
- Create a shared folder on a computer that can be accessed by the servers on which the prerequisites will be installed.
- Copy the files that you downloaded from the Internet to the shared folder.
After you finish creating an available network location for the prerequisites, use the procedure in the following section to install SharePoint 2013 prerequisites on a server.
-
Install the SharePoint 2013 prerequisites at the command prompt
You can install one or more of the prerequisites from the command line using the following procedure.
To install from the command line
- From the Start menu, open the Command Prompt window using the Run as administrator option.
- Navigate to the SharePoint 2013 source directory.
-
Type the prerequisite program switch and corresponding argument for the program that you want to install, and then press ENTER, for example:
PrerequisiteInstaller.exe /SQLNCli: “\\o15-sf-admin\SP_prereqs\sqlncli.msi”
Note:
To install more than one prerequisite, type each switch and argument pair. Be sure to separate each pair by a space, for example:
PrerequisiteInstaller.exe /IDFX: “\\<path>\Windows6.1-KB974405-x64.msu” /sqlncli:”\\<path>\sqlncli.msi” /Sync:”\\<path>\Synchronization.msi”
-
Install the SharePoint 2013 prerequisites by using an arguments file
You can install the prerequisites from the file share using an arguments file that consists of switches and corresponding path statements to the programs that have to be installed.
When you run PrerequisiteInstaller.exe with an arguments file, the following happens:
- PrerequisiteInstaller.exe reads the argument file to verify that each switch is valid and that the program identified in the path statement exists.
Note:
If you specify an argument, PrerequisiteInstaller.exe ignores the arguments file and only processes the command-line argument.
- PrerequisiteInstaller.exe scans the local system to determine whether any of the prerequisites are already installed.
- PrerequisiteInstaller.exe installs the programs in the argument file and returns one of the following exit codes:
- 0 – Success
- 1 – Another instance of this application is already running
- 2 – Invalid command line parameter
- 1001 – A pending restart blocks installation
- 3010 – A restart is needed
- If a prerequisite requires a restart, a 3010 code is generated and you are prompted to click Finish to restart the system. The behavior of the installer after a 3010 code is different depending on which of the following conditions are true on the computer:
- If the component that requires a restart is already installed on the system, the 3010 code is generated and the remaining prerequisites are installed. After the last prerequisite is installed you are prompted to restart the system.
-
If the component that requires a restart is installed on the system by PrerequisiteInstaller.exe, the installer generates the 3010 code, and the installation of the remaining prerequisites is skipped. You are prompted to restart the system.
After the system restarts, PrerequisiteInstaller.exe starts to run again because the startup file that is created before the restart contains a /continue flag.
Multiple components may require a restart. So PrerequisiteInstaller.exe may have to be restarted several times. After a restart, PrerequisiteInstaller.exe ignores the arguments file and attempts to download and install the remaining prerequisites from the Internet. For more information, see Known issues.
Use the following procedure to create an arguments file.
To create an arguments file
- Using a text editor, create a new text document named PrerequisiteInstaller.Arguments.txt. Save this file to the same location as PrerequisiteInstaller.exe. This file will contain the switches and arguments that are used when you run the Microsoft SharePoint Products Preparation Tool.
-
Using a text editor, edit PrerequisiteInstaller.Arguments.txt and provide file paths to the installation source for each prerequisite switch by using the following syntax:
/switch: <path>
Where /switch is a valid switch and <path> is a path of the installation source.
The following example shows a complete arguments file that uses a file share as a common installation point. Do not include carriage returns in your file.
/PowerShell:”<path>\WINDOWS6.1-KB2506143-x64.msu” /NETFX:”<path>\dotNetFx45_Full_x86_x64.exe” /IDFX:”<path>\Windows6.1-KB974405-x64.msu” /sqlncli:”<path>\sqlncli.msi” /Sync:”<path>\Synchronization.msi” /AppFabric:”<path>\setup.exe” /IDFX11:”<path>\Microsoft Identity Extensions.msi” /MSIPCClient:”<path>\msipc.msi” /WCFDataServices:”<path>\WcfDataServices.exe” /KB2671763:”<path>\AppFabric1.1-RTM-KB2671763-x64-ENU.exe
- After you finish editing PrerequisiteInstaller.Arguments.txt, save your edits, and verify that this file is in the same directory as PrerequisiteInstaller.exe.
Use the following procedure to install the prerequisites.
To install the prerequisites using an arguments file
- Run PrerequisiteInstaller.exe at the command prompt to install the prerequisites.
Caution:
If you are prompted to click Finish to restart the system, do not do so. Instead, click Cancel. For more information, see Known issues you continue with the next step.
- Restart the system manually.
-
At the command prompt type the following command and then press Enter:
PrerequisiteInstaller.exe
There are two known issues that affect the use of an arguments file:
-
Using line breaks in the arguments file
If you create an arguments file and use line breaks to put each switch and argument on a separate line, the prerequisite installer fails. The workaround is to enter all the switch and argument pairs on a single line.
-
After a computer restart, the arguments file is not used
After a restart, PrerequisiteInstaller.exe executes the startup command file, which contains a /continue flag. The /continue flag forces the installer to ignore the arguments file.
You must prevent a restart by deleting the startup task in this command file by using one of the following options:
Option 1
- Run PrerequisiteInstaller.exe by double-clicking it. The program will display the first screen with the list of prerequisites.
-
Click Cancel. PrerequisiteInstaller.exe deletes the startup task.
Option 2
- From the Start menu, choose Run and then type regedit to open the registry.
- Open the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders.
- Check the value for “Common Startup”. This shows the directory where the startup tasks are listed.
- Close the registry editor without making any changes.
- Navigate to the startup directory, which is usually <systemdir>\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup.
- Delete the startup task by deleting “SharePointServerPreparationToolStartup_0FF1CE14-0000-0000-0000-000000000000.cmd”.
-
Install SharePoint 2013
Published: July 16, 2012
Summary: Introduces articles that describe how to install SharePoint 2013 in various topologies, on both physical and virtual environments.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Testing and implementing SharePoint 2013 solutions at different stages of the deployment life cycle requires deployments in various topologies.
The following articles on TechNet provide information about how to deploy SharePoint 2013 on one or more servers to create different topologies that you can use for testing and implementing SharePoint 2013 solutions at different stages of the deployment life cycle.
-
TechNet articles about how to install and configure SharePoint 2013
The following articles about how to install and configure SharePoint 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Describes how to install SharePoint 2013 on a single server. This deployment uses SQL Server and can easily be scaled out to create two- and three-tier farm topologies.
Install SharePoint 2013 on a single server with a built-in database
Explains how to install SharePoint 2013 on a single server. This deployment uses SQL Server Express and is typically used for evaluating SharePoint 2013.
Install SharePoint 2013 across multiple servers for a three-tier farm
Describes how to install SharePoint 2013 on multiple servers. This deployment uses SQL Server and the resulting three-tier topology provides the foundation for implementing any solution.
Install and configure a virtual environment for SharePoint 2013
This article describes how to use Windows PowerShell to install SharePoint 2013 in a Hyper-V environment.
Describes language packs and how to download, install, and uninstall them.
Explains how to add a web or application server to a farm. The procedures in this article apply to a SharePoint 2013 farm that consists of at least two tiers. They should not be used for converting a single server deployment to a multiple server farm.
Provides information about how to add a new database server to an existing SharePoint 2013 farm.
Describes how to remove a web server, application server, or a database server from a SharePoint 2013 farm.
Describes how to remove SharePoint 2013 from a computer.
Install and configure a virtual environment for SharePoint 2013
Learn about permissions, accounts, security settings, and what you have to do to prepare your Windows Server 2008 Hyper-V environment for SharePoint 2013.
-
Additional resources about how to install and configure SharePoint 2013
The following resources about how to install and configure SharePoint 2013 are available from other subject matter experts.
Content
Description
Installation and Deployment for SharePoint 2013 Resource Center
Visit the Resource Center to access videos, Community Sites, documentation, and more.
-
Install SharePoint 2013 on a single server with a built-in database
Published: July 16, 2012
Summary: Learn how to install SharePoint 2013 with a built-in database on a single server.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
You can quickly publish a SharePoint site by deploying SharePoint 2013 on a single server that has a built-in database. This configuration is useful if you want to evaluate SharePoint 2013 features and capabilities, such as collaboration, document management, and search. This configuration is also useful if you are deploying only a few websites and you want to minimize administrative overhead.
This article contains required information and procedures to install and configure SharePoint 2013 with a built-in database on a single server.
In this article:
Important:
The steps in this article apply to SharePoint Foundation 2013 and SharePoint Server 2013. The procedures in this topic install Microsoft SQL Server 2008 R2 SP1 Express Edition. However, User Profile synchronization does not work with the Express Edition. If you intend to use User Profile synchronization with SharePoint Server 2013, you must choose a different installation scenario.
-
Overview
When you deploy SharePoint 2013 on a single server that has a built-in database by using the default settings, Setup installs Microsoft SQL Server 2008 R2 SP1 Express Edition and the SharePoint product. The SharePoint Products Configuration Wizard creates the configuration database and content database for the SharePoint sites. Additionally, the SharePoint Products Configuration Wizard installs the SharePoint Central Administration website and creates your first SharePoint site collection.
Note:
This article does not describe how to install SharePoint 2013 in a farm environment, or how to upgrade from previous releases of SharePoint 2013. For more information about how to install SharePoint 2013 on a single-server farm, see Install SharePoint 2013 on a single server with SQL Server. For more information about how to install SharePoint 2013 on a multiple server farm, see Install SharePoint 2013 across multiple servers for a three-tier farm. For more information about upgrade, see Upgrade to SharePoint 2013.
Note:
The Distributed Cache service gives you a complete social computing experience. For more information about the Distributed Cache service, see Overview of microblog features, feeds, and the Distributed Cache service in SharePoint Server 2013, Manage the Distributed Cache service in SharePoint Server 2013, Plan for feeds and the Distributed Cache service (SharePoint Server 2013), and What’s new in authentication for SharePoint 2013
Consider the following restrictions of this method of installation:
- You cannot use this method on a domain controller or in a workgroup environment.
- This method is not supported for production on a domain controller.
- If your computer is in a workgroup, you cannot install AppFabric for Windows Server.
- A Microsoft SQL Server 2008 R2 SP1 Express Edition database cannot be larger than 10 GB.
- You cannot use user profile synchronization in this type of installation. If you want to use user profile synchronization, you must use a server farm installation of SharePoint 2013. For more information, see Install SharePoint 2013 on a single server with SQL Server or Install SharePoint 2013 across multiple servers for a three-tier farm, and Configure profile synchronization (SharePoint 2013).
- You cannot use this method on a domain controller or in a workgroup environment.
-
Before you begin
Before you begin installation, make sure that you have met all hardware and software requirements. For more information, see Hardware and software requirements (SharePoint 2013). To make sure that you perform a clean installation of SharePoint 2013, you must first remove any earlier version of SharePoint 2013 and any pre-release prerequisites if installed.
-
Install SharePoint 2013
To install and configure SharePoint 2013, follow these steps:
- Run the Microsoft SharePoint Products Preparation Tool.
- Run Setup, which installs Microsoft SQL Server 2008 R2 SP1 Express Edition and the SharePoint product.
- Run the SharePoint Products Configuration Wizard, which installs and configures the configuration database, the content database, and installs the SharePoint Central Administration website. This wizard also creates your first SharePoint site collection.
- Configure browser settings.
- Perform post-installation steps.
Important:
To complete the following procedures, you must be a member of the Administrators group on the computer on which you are installing SharePoint 2013.
-
Run the Microsoft SharePoint Products Preparation Tool
Because the prerequisite installer downloads components from the Microsoft Download Center, you must have Internet access on the computer on which you are running the installer. Use the following procedure to install software prerequisites for SharePoint 2013.
To run the Microsoft SharePoint Products Preparation Tool
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- In the folder where you downloaded the SharePoint 2013 software, locate and then run prerequisiteinstaller.exe.
- On the Welcome to the Microsoft SharePoint Products Preparation Tool page, click Next.
- On the License Terms for software products page, review the terms, select the I accept the terms of the License Agreement(s) check box, and then click Next.
- On the Installation Complete page, click Finish.
- After you complete the Microsoft SharePoint Products Preparation Tool, you must also install the following:
- KB 2554876
- KB 2708075
- KB 2759112
-
Run Setup
The following procedure installs Microsoft SQL Server 2008 R2 SP1 Express Edition and the SharePoint product. At the end of Setup, you can choose to start the SharePoint Products Configuration Wizard, which is described later in this section.
To run Setup
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- On the SharePoint Server 2013 or SharePoint Foundation 2013 Start page, click Install SharePoint Server or Install SharePoint Foundation.
- On the Enter Your Product Key page, enter your product key, and then click Continue.
- On the Read the Microsoft Software License Terms page, review the terms, select the I accept the terms of this agreement check box, and then click Continue.
- On the Server Type tab, click Standalone.
- When Setup finishes, a dialog box prompts you to complete the configuration of your server. Ensure that the Run the SharePoint Products Configuration Wizard now check box is selected.
- Click Close to start the configuration wizard.
Note:
If Setup fails, check log files in the Temp folder of the user account that you used to run Setup. Ensure that you are logged in using the same user account, and then type %temp% in the location bar in Windows Explorer. If the path in Windows Explorer resolves to a location that ends in a “1” or “2”, you will have to navigate up one level to view the log files. The log file name is SharePoint Server Setup (<time stamp>).
-
Run the SharePoint Products Configuration Wizard
Use the following procedure to install and configure the configuration database and the content database, and install the SharePoint Central Administration website.
To run the SharePoint Products Configuration Wizard
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- If you have closed the SharePoint Products Configuration Wizard, you can access it by clicking Start, point to All Programs, click SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard. If the User Account Control dialog box appears, click Continue.
- On the Welcome to SharePoint Products page, click Next.
- In the dialog box that notifies you that some services might have to be restarted during configuration, click Yes.
- On the Configuration Successful page, click Finish.
Note:
If the SharePoint Products Configuration Wizard fails, check the PSCDiagnostics log files, which are located on the drive on which SharePoint 2013 is installed, in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder.
- On the Template Selection page, select one of the following options, and then click OK:
- In the Template Selection section, click a predefined template.
- In the Solutions Gallery section, click Solutions Gallery, and customize your own site template.
- On the Set Up Groups for this Site page, specify who should have access to your site, and then either create a new group or use an existing group for these users by doing one of the following:
- To create a new group, click Create a new group, and then type the name of the group and the members that you want to be part of this group.
- To use an existing group, click Use an existing group, and then select the user group in the Item list.
- Click OK.
Note:
If you are prompted for your user name and password, you might have to add the SharePoint Central Administration website to the list of trusted sites and configure user authentication settings in Internet Explorer. You might also want to disable the Internet Explorer Enhanced Security settings. If you see a proxy server error message, you might have to configure proxy server settings so that local addresses bypass the proxy server. For more information about how to configure browser and proxy settings, see Configure browser settings.
After you run the SharePoint Products Configuration Wizard, you should confirm that SharePoint 2013 works correctly by configuring additional settings in Internet Explorer.
If you are not using Internet Explorer, you might have to configure additional settings for your browser. For information about supported browsers, see Plan browser support (SharePoint 2013).
To confirm that you have configured browser settings correctly, log on to the server by using an account that has local administrative credentials. Next, connect to the SharePoint Central Administration website. If you are prompted for your user name and password when you connect, perform the following procedures:
- Add the SharePoint Central Administration website to the list of trusted sites
- Disable Internet Explorer Enhanced Security settings
If you receive a proxy server error message, perform the following procedure:
- Configure proxy server settings to bypass the proxy server for local addresses
To add the SharePoint Central Administration website to the list of trusted sites
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- In Internet Explorer, on the Tools menu, click Internet Options.
- On the Security tab, in the Select a zone to view or change security settings area, click Trusted Sites, and then click Sites.
- Clear the Require server verification (https:) for all sites in this zone check box.
- In the Add this web site to the zone box, type the URL to your site, and then click Add.
- Click Close to close the Trusted Sites dialog box.
- Click OK to close the Internet Options dialog box.
To disable Internet Explorer Enhanced Security settings
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- Click Start, point to All Programs, point to Administrative Tools, and then click Server Manager.
- In Server Manager, select the root of Server Manager.
-
In the Security Information section, click Configure IE ESC.
The Internet Explorer Enhanced Security Configuration dialog box appears.
- In the Administrators section, click Off to disable the Internet Explorer Enhanced Security settings, and then click OK.
To configure proxy server settings to bypass the proxy server for local addresses
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- In Internet Explorer, on the Tools menu, click Internet Options.
- On the Connections tab, in the Local Area Network (LAN) settings area, click LAN Settings.
- In the Automatic configuration area, clear the Automatically detect settings check box.
- In the Proxy Server area, select the Use a proxy server for your LAN check box.
- Type the address of the proxy server in the Address box.
- Type the port number of the proxy server in the Port box.
- Select the Bypass proxy server for local addresses check box.
- Click OK to close the Local Area Network (LAN) Settings dialog box.
- Click OK to close the Internet Options dialog box.
-
Post-installation steps
After you install SharePoint 2013, your browser window opens to the SharePoint Central Administration website of your new SharePoint site. Although you can start to add content to the site or customize the site, we recommend that you first perform the following administrative tasks:
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
- Configure diagnostic logging You can configure diagnostic logging that might be required after initial deployment or upgrade. The default settings are sufficient for most situations, but depending on the business needs and life cycle of the farm, you might want to change these settings. For more information, see Configure diagnostic logging (SharePoint 2013).
- Configure incoming e-mail You can configure incoming e-mail so that SharePoint sites accept and archive incoming e-mail. You can also configure incoming e-mail so that SharePoint sites can archive e-mail discussions as they occur, save e-mailed documents, and show e-mailed meetings on site calendars. In addition, you can configure the SharePoint Directory Management Service to provide support for e-mail distribution list creation and administration. For more information, see Configure incoming email for a SharePoint 2013 farm.
- Configure outgoing e-mail You can configure outgoing e-mail so that your Simple Mail Transfer Protocol (SMTP) server sends e-mail alerts to site users and notifications to site administrators. You can configure both the “From” e-mail address and the “Reply” e-mail address that appear in outgoing alerts. For more information, see Configure outgoing email for a SharePoint 2013 farm.
- Configure search settings You can configure search settings to crawl the content in SharePoint 2013.
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
-
Install SharePoint 2013 on a single server with SQL Server
Published: July 16, 2012
Summary: Learn how to install SharePoint 2013 on a single server.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
A single server installation consists of one server that runs both SQL Server and SharePoint 2013. You can install and configure SharePoint 2013 on a single server if you are hosting only a few sites for a limited number of users or if you want to create a trial or development environment. This configuration is also useful if you want to configure a farm to meet your needs first, and then add servers to the farm at a later stage.
In this article:
- Overview
- Before you install SharePoint 2013 on a single server
- Install SharePoint 2013 on a single server
- Post-installation steps
Important:
The steps in this article apply to SharePoint Foundation 2013 and SharePoint Server 2013.
- Overview
-
Overview
When you install SharePoint 2013 on a single server, you can configure SharePoint 2013 to meet your specific needs. After you have completed setup and the SharePoint Products Configuration Wizard, you will have installed binaries, configured security permissions, configured registry settings, configured the configuration database, configured the content database, and installed the SharePoint Central Administration web site. Next, you can choose to run the Farm Configuration Wizard to configure the farm, select the services that you want to use in the farm, and create the first site collection, or you can manually perform the farm configuration at your own pace.
-
Before you install SharePoint 2013 on a single server
Before you begin to install and configure SharePoint 2013, do the following:
- Ensure that you are familiar with the operating-system guidelines described in Performance Tuning Guidelines for Windows Server 2008 and Performance Tuning Guidelines for Windows Server 2008 R2.
- Ensure that you have met all hardware and software requirements. You must have a 64-bit version of Windows Server 2008 R2 SP1. For server farms, you must also have a 64-bit version of SQL Server 2008 R2 SP1. For more information about these requirements, such as specific updates that you must install, see Hardware and software requirements (SharePoint 2013).
- Ensure that you perform a clean installation of SharePoint 2013.
- Ensure that you are prepared to set up the required accounts by using appropriate permissions. For detailed information, see Initial deployment administrative and service accounts in SharePoint 2013.
- Ensure the Max degree of parallelism is set to 1. For additional information about max degree of parallelism see, Configure the max degree of parallism Server Configuration option and Degree of Parallelism
Note:
The Distributed Cache service gives you a complete social computing experience. For more information about the Distributed Cache service, see Overview of microblog features, feeds, and the Distributed Cache service in SharePoint Server 2013, Manage the Distributed Cache service in SharePoint Server 2013, Plan for feeds and the Distributed Cache service (SharePoint Server 2013), and What’s new in authentication for SharePoint 2013
Security
As a security best practice, we recommend that you install SharePoint 2013 by using least-privilege administration.
- Ensure that you are familiar with the operating-system guidelines described in Performance Tuning Guidelines for Windows Server 2008 and Performance Tuning Guidelines for Windows Server 2008 R2.
-
Install SharePoint 2013 on a single server
To install and configure SharePoint 2013 on a single server, you will follow these steps:
- Run the Microsoft SharePoint Products Preparation Tool, which installs all prerequisites to use SharePoint 2013.
- Run Setup, which installs binaries, configures security permissions, and edits registry settings for SharePoint 2013.
- Run SharePoint Products Configuration Wizard, which installs and configures the configuration database, installs and configures the content database, and installs the SharePoint Central Administration web site.
- Configure browser settings.
- Run the Farm Configuration Wizard, which configures the farm, creates the first site collection, and selects the services that you want to use in the farm.
- Perform post-installation steps.
Important:
To complete the following procedures, the account that you use must be a member of the Administrators group on the computer on which you are installing SharePoint 2013. For information about user accounts, see Initial deployment administrative and service accounts in SharePoint 2013.
-
Run the Microsoft SharePoint Products Preparation Tool
Because the prerequisite installer downloads components from the Microsoft Download Center, you must have Internet access on the computer on which you are running the installer. Use the following procedure to install software prerequisites for SharePoint 2013.
To run the Microsoft SharePoint Products Preparation Tool
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- In the folder where you downloaded the SharePoint 2013 software, locate and then run prerequisiteinstaller.exe.
- On the Welcome to the Microsoft SharePoint Products Preparation Tool page, click Next.
- On the License Terms for software products page, review the terms, select the I accept the terms of the License Agreement(s) check box, and then click Next.
- On the Installation Complete page, click Finish.
- After you complete the Microsoft SharePoint Products Preparation Tool, you must also install the following:
- KB 2554876
- KB 2708075
- KB 2759112
-
Run Setup
The following procedure installs binaries, configures security permissions, and edits registry settings for SharePoint 2013. At the end of Setup, you can choose to start the SharePoint Products Configuration Wizard, which is described later in this section.
To run Setup
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- On the SharePoint Server 2013 Start page, click Install SharePoint Server.
- On the Enter Your Product Key page, enter your product key, and then click Continue.
- On the Read the Microsoft Software License Terms page, review the terms, select the I accept the terms of this agreement check box, and then click Continue.
-
On the Server Type tab, click Complete.
The stand-alone option is used to install a single server that has a built-in database.
- Optional: To install SharePoint 2013 at a custom location, click the File Location tab, and then either type the location or click Browse to find the location.
- Click Install Now.
- When Setup finishes, a dialog box prompts you to complete the configuration of your server. Ensure that the Run the SharePoint Products and Technologies Configuration Wizard now check box is selected.
- Click Close to start the configuration wizard.
Note:
If Setup fails, check log files in the Temp folder of the user account you used to run Setup. Ensure that you are logged in using the same user account and then type %temp% in the location bar in Windows Explorer. If the path in Windows Explorer resolves to a location that ends in a “1” or “2”, you have to navigate up one level to view the log files. The log file name is SharePoint Server Setup (<time stamp>).
-
Run the SharePoint Products Configuration Wizard
Use the following procedure to install and configure the configuration database and the content database, and to install the SharePoint Central Administration website.
To run the SharePoint Products Configuration Wizard
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- If you have closed the SharePoint Products Configuration Wizard, you can access it by clicking Start, point to All Programs, click SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard. If the User Account Control dialog box appears, click Continue.
- On the Welcome to SharePoint Products page, click Next.
- In the dialog box that notifies you that some services might have to be restarted during configuration, click Yes.
- On the Connect to a server farm page, click Create a new server farm, and then click Next.
-
On the Specify Configuration Database Settings page, do the following:
- In the Database server box, type the name of the computer that is running SQL Server.
- In the Database name box, type a name for your configuration database or use the default database name. The default name is SharePoint_Config.
- In the Username box, type the user name of the server farm account. Ensure that you type the user name in the format DOMAIN\user name.
- In the Database server box, type the name of the computer that is running SQL Server.
Security
The server farm account is used to create and access your configuration database. It also acts as the application pool identity account for the SharePoint Central Administration application pool, and it is the account under which the Microsoft SharePoint Foundation Workflow Timer service runs. The SharePoint Products Configuration Wizard adds this account to the SQL Server Login accounts, the SQL Serverdbcreator server role, and the SQL Serversecurityadmin server role. The user account that you specify as the service account has to be a domain user account. However, it does not have to be a member of any specific security group on your front-end web servers or your database servers. We recommend that you follow the principle of least-privilege and specify a user account that is not a member of the Administrators group on your front-end web servers or your database servers.
- In the Password box, type the user password.
- Click Next.
-
On the Specify Farm Security Settings page, type a passphrase, and then click Next.
Although a passphrase resembles a password, it is usually longer to improve security. It is used to encrypt credentials of accounts that are registered in SharePoint 2013. For example, the SharePoint 2013 system account that you provide when you run the SharePoint Products Configuration Wizard. Ensure that you remember the passphrase, because you must use it every time that you add a server to the farm.
Ensure that the passphrase meets the following criteria:
- Contains at least eight characters
-
Contains at least three of the following four character groups:
- English uppercase characters (from A through Z)
- English lowercase characters (from a through z)
- Numerals (from 0 through 9)
- Nonalphabetic characters (such as !, $, #, %)
- English uppercase characters (from A through Z)
-
On the Configure SharePoint Central Administration Web Application page, do the following:
- Either select the Specify port number check box and type the port number that you want the SharePoint Central Administration web application to use, or leave the Specify port number check box cleared if you want to use the default port number.
- Click either NTLM or Negotiate (Kerberos).
- Either select the Specify port number check box and type the port number that you want the SharePoint Central Administration web application to use, or leave the Specify port number check box cleared if you want to use the default port number.
- Click Next.
- After you complete the SharePoint Products Configuration Wizard page, review your configuration settings to verify that they are correct, and then click Next.
Note:
The Advanced Settings option is not available in SharePoint 2013.
-
On the Configuration Successful page, click Finish. When the wizard closes, setup opens the web browser and connects to Central Administration.
If the SharePoint Products Configuration Wizard fails, check the PSCDiagnostics log files, which are located on the drive on which SharePoint 2013 is installed, in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder.
If you are prompted for your user name and password, you might have to add the SharePoint Central Administration web site to the list of trusted sites and configure user authentication settings in Internet Explorer. You might also want to disable the Internet Explorer Enhanced Security settings. If you see a proxy server error message, you might have to configure proxy server settings so that local addresses bypass the proxy server. Instructions for configuring proxy server settings are provided in the following section. For more information about how to configure browser and proxy settings, see Configure browser settings.
-
Configure browser settings
After you run the SharePoint Products Configuration Wizard, you should confirm that SharePoint 2013 works correctly by configuring additional settings in Internet Explorer.
If you are not using Internet Explorer, you might have to configure additional settings for your browser. For information about supported browsers, see Plan browser support (SharePoint 2013).
To confirm that you have configured browser settings correctly, log on to the server by using an account that has local administrative credentials. Next, connect to the SharePoint Central Administration web site. If you are prompted for your user name and password when you connect, perform the following procedures:
- Add the SharePoint Central Administration website to the list of trusted sites
- Disable Internet Explorer Enhanced Security settings
If you receive a proxy server error message, perform the following procedure:
- Configure proxy server settings to bypass the proxy server for local addresses
To add the SharePoint Central Administration website to the list of trusted sites
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- In Internet Explorer, on the Tools menu, click Internet Options.
- On the Security tab, in the Select a zone to view or change security settings area, click Trusted Sites, and then click Sites.
- Clear the Require server verification (https:) for all sites in this zone check box.
- In the Add this web site to the zone box, type the URL to your site, and then click Add.
- Click Close to close the Trusted Sites dialog box.
- Click OK to close the Internet Options dialog box.
To disable Internet Explorer Enhanced Security settings
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- Click Start, point to All Programs, point to Administrative Tools, and then click Server Manager.
- In Server Manager, select the root of Server Manager.
-
In the Security Information section, click Configure IE ESC.
The Internet Explorer Enhanced Security Configuration dialog box appears.
- In the Administrators section, click Off to disable the Internet Explorer Enhanced Security settings, and then click OK.
To configure proxy server settings to bypass the proxy server for local addresses
- Verify that the user account that completes this procedure has the following credentials:
- The user account is a member of the Administrators group on the computer on which you are performing the procedure.
- In Internet Explorer, on the Tools menu, click Internet Options.
- On the Connections tab, in the Local Area Network (LAN) settings area, click LAN Settings.
- In the Automatic configuration area, clear the Automatically detect settings check box.
- In the Proxy Server area, select the Use a proxy server for your LAN check box.
- Type the address of the proxy server in the Address box.
- Type the port number of the proxy server in the Port box.
- Select the Bypass proxy server for local addresses check box.
- Click OK to close the Local Area Network (LAN) Settings dialog box.
- Click OK to close the Internet Options dialog box.
-
Run the Farm Configuration Wizard
You have now completed setup and the initial configuration of SharePoint 2013. You have created the SharePoint Central Administration web site. You can now create your farm and sites, and you can select services by using the Farm Configuration Wizard.
To run the Farm Configuration Wizard
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- On the SharePoint Central Administration home page, on the Quick Launch, click Configuration Wizards, and then click Launch the Farm Configuration Wizard.
- On the Help Make SharePoint Better page, click one of the following options, and then click OK:
- Yes, I am willing to participate (Recommended.)
- No, I don’t want to participate.
- On the Configure your SharePoint farm page, next to Yes, walk me through the configuration of my farm using this wizard, click Start the Wizard.
- On the Configure your SharePoint farm page, in the Service Account section, click the service account option that you want to use to configure your services.
Security
For security reasons, we recommend that you use a different account from the farm administrator account to configure services in the farm.
If you decide to use an existing managed account — that is, an account of which SharePoint 2013 is aware — make sure that you click that option before you continue.
- In the Services section, review the services that you want to use in the farm, and then click Next.
Note:
For more information, see Configure services and service applications in SharePoint 2013. If you are using Office Web Apps, see Office Web Apps (SharePoint 2013).
-
On the Create Site Collection page, do the following:
- In the Title and Description section, in the Title box, type the name of your new site.
- Optional: In the Description box, type a description of what the site contains.
- In the Web Site Address section, select a URL path for the site.
- In the Template Selection section, in the Select a template list, select the template that you want to use for the top-level site in the site collection.
- In the Title and Description section, in the Title box, type the name of your new site.
Note:
To view a template or a description of a template, click any template in the Select a template list.
- Click OK.
- On the Configure your SharePoint farm page, review the summary of the farm configuration, and then click Finish.
-
Post-installation steps
After you install and configure SharePoint 2013, your browser window opens to the Central Administration web site of your new SharePoint site. Although you can start adding content to the site or customizing the site, we recommend that you first perform the following administrative tasks.
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
- Configure diagnostic logging You can configure diagnostic logging that might be required after initial installation or upgrade. The default settings are sufficient for most situations. Depending upon the business needs and life-cycle of the farm, you might want to change these settings. For more information, see Configure diagnostic logging (SharePoint 2013).
- Configure incoming e-mail You can configure incoming e-mail so that SharePoint sites accept and archive incoming e-mail. You can also configure incoming e-mail so that SharePoint sites can archive e-mail discussions as they occur, save e-mailed documents, and show e-mailed meetings on site calendars. In addition, you can configure the SharePoint Directory Management Service to provide support for e-mail distribution list creation and administration. For more information, see Configure incoming email for a SharePoint 2013 farm.
- Configure outgoing email You can configure outgoing email so that your Simple Mail Transfer Protocol (SMTP) server sends email alerts to site users and notifications to site administrators. You can configure both the “From” email address and the “Reply” email address that appear in outgoing alerts. For more information, see Configure outgoing email for a SharePoint 2013 farm.
- Configure Search settings You can configure Search settings to crawl the content in SharePoint 2013.
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
-
Install SharePoint 2013 across multiple servers for a three-tier farm
Published: July 16, 2012
Summary: Learn how to install SharePoint 2013 to create a server farm that includes web servers, an application server, and a database server.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
A three-tier farm configuration consists of two front-end web servers, an application server, and a database server. The deployment sequence and configurations that are described in this article are based on recommended best practices. While the farm configuration is not complex, it provides a fundamental infrastructure to implement a SharePoint 2013 solution on similar — or more complex farms.
In this article:
-
Overview
The basic steps in this deployment are as follows:
- Ensure that you are familiar with the concept of a three-tier topology.
- Ensure that you have done all the planning and preparatory work, such as verifying hardware and software requirements.
- Install the required software updates on all servers that will be part of the farm.
- Install the SharePoint 2013 prerequisites on servers in the application and web tiers.
- Install SharePoint 2013 on the application server and the web servers.
- Create and configure the SharePoint farm.
- Provision services.
- Complete post-deployment tasks as required.
Topology overview
This topology is typically used for the medium and large farms described in Overview of SharePoint 2013 installation and configuration. In terms of performance, capacity, and scalability, a three-tier topology is recommended over a two-tier topology. A three-tier topology provides the most efficient physical and logical layout to support scaling out or scaling up, and it provides better distribution of services across the member servers of the farm. The following illustration shows the three-tier deployment that is described in this article.
Three-tier farm configuration
In the previous illustration, note the following:
- You can add web servers to the web tier. These servers can be configured as conventional web servers to handle user requests, or they can be configured to host dedicated query components or other service components.
- You can add farm servers to the application tier and configure them as dedicated servers that will host the SharePoint Central Administration website or other services on the farm that require dedicated resources or isolation from the web tier — for example, crawl components, query components, and profile pages.
-
You can add database servers to the database tier to implement a stand-alone instance, database mirroring, or a failover cluster. To configure the farm for high availability, database mirroring or a failover cluster is required on the database tier.
-
Before you install SharePoint 2013 on multiple servers for a three-tier farm
-
Before you begin to install and configure SharePoint 2013, do the following:
- Ensure that you are familiar with the operating-system guidelines described in Performance Tuning Guidelines for Windows Server 2008 and Performance Tuning Guidelines for Windows Server 2008 R2.
- Ensure that you have met all hardware and software requirements. You must have a 64-bit version of Windows Server 2008 R2 SP1. For server farms, you must also have a supported 64-bit version of SQL Server. For more information about these requirements, such as specific updates that you must install, see Hardware and software requirements (SharePoint 2013).
- Ensure that you perform a clean installation of SharePoint 2013.
- Ensure that you are prepared to set up the required accounts by using appropriate permissions. For detailed information, see Initial deployment administrative and service accounts in SharePoint 2013.
Note:
If your computer is in a Workgroup, you cannot install AppFabric for Windows Server.
Note:
The Distributed Cache service gives you a complete social computing experience. For more information about the Distributed Cache service, see Overview of microblog features, feeds, and the Distributed Cache service in SharePoint Server 2013, Manage the Distributed Cache service in SharePoint Server 2013, Plan for feeds and the Distributed Cache service (SharePoint Server 2013), and What’s new in authentication for SharePoint 2013
-
Using the Microsoft SharePoint Products Preparation Tool
The Microsoft SharePoint Products Preparation Tool checks for the presence of prerequisites, and installs and configures all required programs. The Microsoft SharePoint Products Preparation Tool requires an Internet connection to download and configure SharePoint 2013 prerequisites. The Microsoft SharePoint Products Preparation Tool runs when you start to install SharePoint 2013.
-
Database server
Ensure that SQL Server is updated to the required level and the TCP/IP protocol is enabled for the network configuration.
Organizations whose database administrators operate independently from SharePoint administrators will have to make sure that the correct version of SQL Server is available and updated to the required level. In addition, you will have to request a DBA-created database that is configured for your farm.
Ensure the Max degree of parallelism is set to 1. For additional information about max degree of parallelism see, Configure the max degree of parallism Server Configuration option and Degree of Parallelism.
-
Public updates and hotfix packages
Ensure that public updates and the required hotfix packages are installed for the operating system, SQL Server, and SharePoint 2013. We recommend that all servers be updated to the same software version before you apply the public updates.
- Ensure that you are familiar with the concept of a three-tier topology.
-
Prepare the farm servers
Before you install SharePoint 2013 , you must check for and install all the prerequisites on the application server and the web servers by using the Microsoft SharePoint Products Preparation Tool.
Tip:
If you decide to install prerequisites manually, you can still run the Microsoft SharePoint Products Preparation Tool to verify which prerequisites are required on each server.
Use the following procedure to install prerequisites on each server in the farm.
To run the Microsoft SharePoint Products Preparation Tool
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- In the folder where you downloaded the SharePoint 2013 software, locate and then run prerequisiteinstaller.exe.
- On the Welcome to the Microsoft SharePoint Products Preparation Tool page, click Next.
Note:
The preparation tool may have to restart the local server to complete the installation of some prerequisites. The installer will continue to run after the server is restarted without manual intervention. However, you will have to log on to the server again.
- On the License Terms for software products page, review the terms, select the I accept the terms of the License Agreement(s) check box, and then click Next.
- On the Installation Complete page, click Finish.
- After you complete the Microsoft SharePoint Products Preparation Tool, you must also install the following:
-
Install SharePoint 2013 on the farm servers
After the prerequisites are installed, follow these steps to install SharePoint 2013 on each farm server.
The following procedure installs binaries, configures security permissions, and edits registry settings for SharePoint 2013. At the end of Setup, you can choose to start the SharePoint Products Configuration Wizard, which is described later in this article.
To run Setup
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- On the SharePoint 2013 Start page, click Install SharePoint Server.
- On the Enter Your Product Key page, enter your product key, and then click Continue.
- On the Read the Microsoft Software License Terms page, review the terms, select the I accept the terms of this agreement check box, and then click Continue.
- On the Choose the installation you want page, click Server Farm.
- On the Server Type tab, click Complete.
- On the File Location tab, accept the default location or change the installation path, and then click Install Now.
Note:
As a best practice, we recommend that you install SharePoint 2013 on a non-system drive.
- When the Setup program is finished, a dialog box prompts you to complete the configuration of your server. Clear the Run the SharePoint Products and Technologies Configuration Wizard now check box.
Note:
For consistency of approach, we recommend that you do not run the configuration wizard until you have installed SharePoint 2013 all application and front-end web servers that will participate in the server farm.
- Click Close to finish Setup.
-
Create and configure the farm
To create and configure the farm, you run the SharePoint Products Configuration Wizard. This wizard automates several configuration tasks, such as creating the configuration database, installing services, and creating the Central Administration website. We recommend that you run the SharePoint Products Configuration Wizard on the server that will host the SharePoint Central Administration website before you run the wizard on the other servers in the farm.
To run the SharePoint Products Configuration Wizard and configure the farm
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- On the server that will host Central Administration (the application server), click Start, point to All Programs, and then click SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard. If the User Account Control dialog box appears, click Continue.
- On the Welcome to SharePoint Products page, click Next.
- In the dialog box that notifies you that some services might have to be restarted during configuration, click Yes.
- On the Connect to a server farm page, click Create a new server farm, and then click Next.
-
On the Specify Configuration Database Settings page, do the following:
- In the Database server box, type the name of the computer that is running SQL Server.
- In the Database name box, type a name for your configuration database, or use the default database name. The default name is SharePoint_Config.
- In the Username box, type the user name of the server farm account in DOMAIN\user name format.
- In the Database server box, type the name of the computer that is running SQL Server.
Important:
The server farm account is used to create and access your configuration database. It also acts as the application pool identity account for the SharePoint Central Administration application pool, and it is the account under which the SharePoint Timer service runs. The SharePoint Products Configuration Wizard adds this account to the SQL Server Login accounts, the SQL Serverdbcreator server role, and the SQL Serversecurityadmin server role. The user account that you specify as the service account has to be a domain user account. However, it does not have to be a member of any specific security group on your web servers or your database servers. We recommend that you follow the principle of least-privilege, and specify a user account that is not a member of the Administrators group on your front-end web servers or your database servers.
- In the Password box, type the user password.
- Click Next.
-
On the Specify Farm Security Settings page, type a passphrase, and then click Next.
Although a passphrase resembles a password, it is usually longer to improve security. It is used to encrypt credentials of accounts that are registered in SharePoint 2013. For example, the SharePoint 2013 system account that you provide when you run the SharePoint Products Configuration Wizard. Ensure that you remember the passphrase, because you must use it every time that you add a server to the farm.
Ensure that the passphrase meets the following criteria:
- Contains at least eight characters
-
Contains at least three of the following four character groups:
- English uppercase characters (from A through Z)
- English lowercase characters (from a through z)
- Numerals (from 0 through 9)
- Nonalphabetic characters (such as !, $, #, %)
- English uppercase characters (from A through Z)
-
On the Configure SharePoint Central Administration Web Application page, do the following:
- Either select the Specify port number check box and type the port number that you want the SharePoint Central Administration web application to use, or leave the Specify port number check box cleared if you want to use the default port number.
- Either select the Specify port number check box and type the port number that you want the SharePoint Central Administration web application to use, or leave the Specify port number check box cleared if you want to use the default port number.
Note:
If you want to access the SharePoint Central Administration website from a remote computer, make sure that you allow access to the port number that you configure in this step. You do this by configuring the inbound rule for SharePoint Central Administration v4 in Windows Firewall with Advanced Security.
- Click either NTLM or Negotiate (Kerberos).
- Click Next.
- On the Completing the SharePoint Products Configuration Wizard page, click Next.
- On the Configuration Successful page, click Finish.
Note:
If the SharePoint Products Configuration Wizard fails, check the log files on the drive on which SharePoint 2013 is installed, which are located in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder.
-
The Central Administration website will open in a new browser window.
On the Help Make SharePoint Better page, click one of the following options and then click OK.
- Yes, I am willing to participate (Recommended).
- No, I don’t wish to participate.
- Yes, I am willing to participate (Recommended).
-
On the Initial Farm Configuration Wizard page, you have the option to use a wizard to configure services or you can decide to configure services manually. For the purpose of this article, we use the manual option. Click Cancel.
The choice that you make here is a matter of personal preference. The Farm Configuration Wizard will configure some services automatically when you run it. However, if you configure services manually, you have greater flexibility in designing your logical architecture.
For information about how to use the wizard to configure services, see Configure services and service applications in SharePoint 2013. If you are using Microsoft Office Web Apps, see Office Web Apps overview (Installed on SharePoint 2013).
Important:
If you are using a DBA-created database, you cannot use the Farm Configuration Wizard, you must use SharePoint Products Configuration Wizard.
-
Add web servers to the farm
After you create the farm on the application server, you can add the servers for the web tier by following the same process described earlier in this topic for installing SharePoint 2013 on the server that hosts Central Administration. The only difference is that during setup, you are prompted to join an existing farm. Follow the wizard steps to join the farm.
For additional information about how to add servers to a farm, see Add web or application servers to farms in SharePoint 2013. This article also provides detailed information for the steps in the following procedure.
-
Post-installation steps
After you install and configure SharePoint 2013, your browser window opens to the Central Administration web site of your new SharePoint site. Although you can start adding content to the site or customizing the site, we recommend that you first perform the following administrative tasks.
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
- Configure diagnostic logging You can configure diagnostic logging that might be required after initial installation or upgrade. The default settings are sufficient for most situations. Depending upon the business needs and life-cycle of the farm, you might want to change these settings. For more information, see Configure diagnostic logging (SharePoint 2013).
- Configure incoming e-mail You can configure incoming e-mail so that SharePoint sites accept and archive incoming e-mail. You can also configure incoming e-mail so that SharePoint sites can archive e-mail discussions as they occur, save e-mailed documents, and show e-mailed meetings on site calendars. In addition, you can configure the SharePoint Directory Management Service to provide support for e-mail distribution list creation and administration. For more information, see Configure incoming email for a SharePoint 2013 farm.
- Configure outgoing email You can configure outgoing email so that your Simple Mail Transfer Protocol (SMTP) server sends email alerts to site users and notifications to site administrators. You can configure both the “From” email address and the “Reply” email address that appear in outgoing alerts. For more information, see Configure outgoing email for a SharePoint 2013 farm.
- Configure Search settings You can configure Search settings to crawl the content in SharePoint 2013.
- Configure usage and health data collection You can configure usage and health data collection in your server farm. The system writes usage and health data to the logging folder and to the logging database. For more information, see Configure usage and health data collection (SharePoint 2013).
-
Install or uninstall language packs for SharePoint 2013
Published: July 16, 2012
Summary: Learn how to download, install, and uninstall language packs for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Language packs enable site owners and site collection administrators to create SharePoint sites and site collections in multiple languages without requiring separate installations of SharePoint 2013. You install language packs, which contain language-specific site templates, on web and application servers. When an administrator creates a site or a site collection that is based on a language-specific site template, the text that appears on the site or the site collection is displayed in the site template’s language. Language packs are typically used in multinational deployments where a single server farm supports users in different locations, or when sites and web pages must be duplicated in one or more languages.
If users are accessing Project Server 2013 in the SharePoint farm and have to view their project data in another language, they will also have to install a corresponding Project Server 2013 language pack. For more information about Project Server 2013 language packs, see Deploy language packs in Project Server 2013
Word breakers and stemmers enable you to search efficiently and effectively across content on SharePoint sites and site collections in multiple languages without requiring separate installations of SharePoint 2013. Word breakers and stemmers are automatically installed on web and application servers by Setup.
Important:
If you are uninstalling SharePoint 2013, you must uninstall all language packs before you uninstall SharePoint 2013.
In this article:
-
About language IDs and language packs
Site owners or site collection administrators who create sites or site collections can select a language for each site or site collection.
The language that they select has a language identifier (ID). The language ID determines the language that is used to display and interpret text that is on the site or site collection. For example, when a site owner creates a site in French, the site’s toolbars, navigation bars, lists, and column headings appear in French. Similarly, if a site owner creates a site in Arabic, the site’s toolbars, navigation bars, lists, and column headings appear in Arabic. In addition, the default left-to-right orientation of the site changes to a right-to-left orientation to correctly display Arabic text.
The language packs that are installed on the web and application servers determine the list of available languages that you can use to create a site or site collection. By default, sites and site collections are created in the language in which SharePoint 2013 was installed. For example, if you install the Spanish version of SharePoint 2013, the default language for sites, site collections, and web pages is Spanish. If someone has to create sites, site collections, or web pages in a language other than the default SharePoint 2013 language, you must install the language pack for that language on the web and application servers. For example, if you are running the French version of SharePoint 2013, and a site owner wants to create sites in French, English, and Spanish, you must install the English and Spanish language packs on the web and application servers.
By default, when a site owner creates a new web page in a site, the site displays text in the language that is specified by the language ID.
Language packs are not bundled into multilingual installation packages. You must install a specific language pack for each language that you want to support. Also, language packs must be installed on each web and application server to make sure that that each web and application server can display content in the specified language.
Important:
You cannot change an existing site, site collection, or web page from one language to another by applying different language-specific site templates. After you use a language-specific site template for a site or a site collection, the site or site collection always displays content in the language of the original site template.
Only a limited set of language packs are available for SharePoint 2013.
Although a site owner specifies a language ID for a site, some user interface elements such as error messages, notifications, and dialog boxes do not display in the language that was specified. This is because SharePoint 2013 relies on several supporting technologies — for example, the Microsoft .NET Framework, Microsoft Windows Workflow Foundation, Microsoft ASP.NET, and SQL Server — some of which are localized into only a limited number of languages. If a user interface element is generated by any of the supporting technologies that are not localized into the language that the site owner specified for the site, the user interface element appears in English. For example, if a site owner creates a site in Hebrew, and the .NET Framework component displays a notification message, the notification message will not display in Hebrew because the .NET Framework is not localized into Hebrew. This situation can occur when sites are created in any language except the following: Chinese, French, German, Italian, Japanese, Korean, and Spanish.
Each language pack that you install creates a folder at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LAYOUTS\Locale_ID that contains language-specific data. In each locale_ID folder, you must have only one HTML error file that contains the error information that is used when a file cannot be found. Anytime a file cannot be found for any site in that language, this file will be used. You can specify the file to use by setting the FileNotFoundPage for each web application.
In some cases, some text might originate from the original installation language, which can create a mixed-language experience. This kind of mixed-language experience is typically seen only by content creators or site owners and is not seen by site users.
-
Downloading language packs
Follow these steps for each language that you want to support. If you decide to download more than one language, please be aware that a unique file that has a common name is downloaded for each language. Therefore, make sure that you download each language pack to a separate folder on the hard disk so that you do not overwrite a language pack of a different language.
Important:
By default, the Windows PowerShell Help files are installed in English (en-us). To view these files in the same language as the operating system, install the language pack for the same language in which the operating system was installed.
You can download language packs from the same location where you downloaded SharePoint 2013.
-
Installing language packs on the web and application servers
After you install the necessary language files on the web and application servers, you can install the language packs. Language packs are available as individual downloads (one download for each supported language). If you have a server farm environment and you are installing language packs to support multiple languages, you must install the language packs on each web and application server.
Important:
The language pack is installed in its native language. The procedure that follows is for the English language pack.
To install a language pack
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- In the folder where you downloaded the language pack, run setup.exe.
- On the Read the Microsoft Software License Terms page, review the terms, select the I accept the terms of this agreement check box, and then click Continue.
- The Setup wizard runs and installs the language pack.
-
Rerun the SharePoint Products Configuration Wizard by using the default settings. If you do not run the SharePoint Products Configuration Wizard after you install a language pack, the language pack will not be installed correctly.
The SharePoint Products Configuration Wizard runs in the language of the base installation of SharePoint 2013, not in the language of the language pack that you just installed.
To rerun the SharePoint 2013 Configuration Wizard
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- Click Start, point to All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard.
- On the Welcome to SharePoint Products page, click Next.
- Click Yes in the dialog box that alerts you that some services might have to be restarted during configuration.
- On the Modify Server Farm Settings page, click Do not disconnect from this server farm, and then click Next.
- If the Modify SharePoint Central Administration Web Administration Settings page appears, do not change any of the default settings, and then click Next.
- After you complete the Completing the SharePoint Products and Technologies Configuration Wizard, click Next.
- On the Configuration Successful page, click Finish.
- After you install a new language pack and rerun the Rerun the SharePoint 2013 Configuration Wizard, you must deactivate and then reactivate any language-specific features before you use the new language pack.
When you install language packs, the language-specific site templates are installed in the %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\TEMPLATE\LanguageID directory, where LanguageID is the Language ID number for the language that you are installing. For example, the United States English language pack installs to the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\TEMPLATE\1033 directory. After you install a language pack, site owners and site collection administrators can create sites and site collections based on the language-specific site templates by specifying a language when they are creating a new SharePoint site or site collection.
-
Uninstalling language packs
If you no longer have to support a language for which you have installed a language pack, you can remove the language pack by using the Control Panel. Removing a language pack removes the language-specific site templates from the computer. All sites that were created that have those language-specific site templates will no longer work (the URL will produce a HTTP 500 – Internal server error page). Reinstalling the language pack will make the site functional again.
You cannot remove the language pack for the version of SharePoint 2013 that you have installed on the server. For example, if you are running the Japanese version of SharePoint 2013, you cannot uninstall the Japanese language support for SharePoint 2013.
-
Add web or application servers to farms in SharePoint 2013
Published: July 16, 2012
Summary: Learn how to add a server to an existing SharePoint 2013 farm so the server can later be configured for use as a front-end web server or as an application server.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The procedures in this article apply to a SharePoint 2013 farm that consists of at least two tiers. This article does not describe how to convert a single-server deployment to a multiple-server farm.
In this article:
-
Before you add a web or application server to a SharePoint farm
Note:
Administrators typically use the SharePoint Central Administration website and the SharePoint Management Shell to manage deployments. For information about accessibility for administrators, see Accessibility for SharePoint Products.
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products
- Keyboard shortcuts
- Touch
-
Determine server role
To add a new server to the farm, you must know its intended role to plan for additional or specialized configurations and assess the potential effect of adding the server to a production environment.
Note:
A typical three-tier farm includes front-end web servers, an application server that also hosts Central Administration, and a database server. The scope of this article is the front-end web server and application server roles.
After you determine the role of the server in your farm topology, you must identify the services and features that must be configured for the server to meet this role. This information will determine how SharePoint 2013 is configured to provision the server for its role in either the web tier or the application tier. For more information, see Manage service applications in SharePoint 2013.
The following illustration shows a SharePoint 2013 farm with two front-end web servers (Web-1 and Web-2) that serve content and host the search query component. The only application server (App-1) hosts Central Administration and the search crawl component for the farm.
Options for adding a server to a farm
The following sections provide information about the general characteristics of the front-end web server and application server roles.
-
Front-end web server role
The fundamental role of a front-end web server is to host web pages, web services, and the Web Parts that are required to process requests from users. The web server directs these requests to the application server, which returns the results to the front-end web server.
Depending on farm requirements, the front-end web server may also be configured to support search in scenarios where there are no dedicated search servers.
Note:
Distributing search is not an option for SharePoint 2013, where only a single search instance is permitted for each content database.
SharePoint 2013 provides more flexibility by letting you to install different search components, typically query components, on one or more front-end web servers. This is option A in the previous farm illustration. A third server also improves load balancing and increases front-end web server availability. Three servers on the web-tier is called a stretched farm.
-
Application server role
By default, the server that hosts Central Administration in a three-tier farm is an application server. You can add application servers to host services that can be deployed to a single server and used by all the servers in a farm.
Services with similar usage and performance characteristics can be logically grouped on a server, and if it is necessary, hosted on multiple servers if a scale out is required to respond to performance or capacity requirements. For example, client-related farm services such as Word Services and Word Viewer can be combined into a service group and hosted on a dedicated server. In addition, some services, such as the Managed Metadata service, can be configured as service application that can be used by other farms.
In the farm illustration, there are two options to add an application server.
- In option B an additional server is configured to host all queries for the farm. The query component is removed from the front-end web servers.
- In option C an additional server is configured as a dedicated crawl server, which offloads farm indexing from the server that hosts Central Administration. The front-end web servers continue to host the query component for the farm.
In a three-tier farm that is running enterprise search, dedicated application servers are typically configured to host individual enterprise search components. Servers hosting a query component are known as query servers and servers hosting a crawl component are known as index servers. For more information, see Manage search topology (SharePoint Server 2013).
-
Additional tasks
Before you start to install prerequisite software, you have to complete the following:
- Verify that the new server meets the hardware and software requirements described in Hardware and software requirements for SharePoint 2013.
- Verify that you have the minimum level of permissions that are required to install and configure SharePoint 2013 on a new server. You must be a member of the Farm Administrators SharePoint group and the Administrators group on the local server to complete the procedures in this article. For more information, see Initial deployment administrative and service accounts in SharePoint 2013.
- Verify that you know the name of the database server on the farm to which you are connecting, and the name of the configuration database if you are adding the server by using Windows PowerShell commands.
- If you intend to use Windows PowerShell commands to add the server, verify that you meet the following minimum memberships: SharePoint 2013 is installed.
- Securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
- An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Document the location of the SharePoint 2013 binary and log files on the existing farm servers. We recommend that the location of these files on the new server map to the locations used on the other servers in the farm. For more information, see Configure diagnostic logging in SharePoint 2013.
Important:
If you change the location of the trace log to a non-system drive, change the location on all the servers in the farm. Existing or new servers cannot log data if the location does not exist. In addition, you will be unable to add new servers unless the path that you specify exists on the new server. You cannot use a network share for logging purposes.
- Plan browser support
-
Install prerequisite software
Before you can install SharePoint 2013 and add a server to the farm, you must check for and install all the prerequisite software on the new server. You do this by using the Microsoft SharePoint Products Preparation Tool, which requires an Internet connection to download and configure SharePoint 2013 prerequisites. If you do not have an Internet connection for the farm servers, you can still use the tool to determine the software that is required. You will have to obtain installable images for the required software. For download locations, see Access to applicable software in “Hardware and software requirements (SharePoint 2013).”
Tip:
After you obtain a copy of the required software, we recommend that you create an installation point that you can use to store the images. You can use this installation point to install future software updates.
For detailed instructions about how to install the prerequisites, see Prepare the farm servers in the article, Install SharePoint 2013 across multiple servers for a three-tier farm.
-
Install the SharePoint software
After you install the prerequisites, follow these steps to install SharePoint 2013 on the new server. For detailed instructions about how to install SharePoint 2013, see Install SharePoint 2013 on a single server with SQL Server.
To install SharePoint 2013
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- From the product media or a file share that contains the SharePoint 2013 Products installation files, run Setup.exe.
- On the Start page, click the link to install SharePoint 2013.
- Review and accept the Microsoft License Terms.
- On the Server Type tab, select Complete.
Note:
You can choose to install only the components that are required for a front-end web server. However, if you perform a complete installation, you have more flexibility to re-purpose the server role in the farm in the future.
- Accept the default file location where SharePoint 2013 will be installed or change the installation path in order to suit your requirements.
Tip:
As a best practice, we recommend that you install SharePoint 2013 on a drive that does not contain the operating system.
- When Setup finishes, a dialog box prompts you to run the SharePoint Products Configuration Wizard. You can start the wizard immediately or from the Windows command prompt later.
-
Add the new SharePoint server to the farm
You add the new server to the farm by using one of the following procedures:
- To add a new SharePoint 2013 server to the farm by using the SharePoint Products Configuration Wizard
- To add a new SharePoint 2013 server to the farm by using Windows PowerShell
To add a new SharePoint 2013 server to the farm by using the SharePoint Products Configuration Wizard
- To add a new SharePoint 2013 server to the farm by using the SharePoint Products Configuration Wizard
- Verify that the user account that is performing this procedure is the Setup user account. For information about the Setup user account, see Initial deployment administrative and service accounts in SharePoint 2013.
- Start the SharePoint 2013 Products Configuration Wizard.
-
For Windows Server 2008 R2:
- On the new server, click Start, point to All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard.
- On the new server, click Start, point to All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Products Configuration Wizard.
-
For Windows Server 2012:
-
On the new server, on the Start screen, click SharePoint 2013 Products Configuration Wizard.
If SharePoint 2013 Products Configuration Wizard is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Products Configuration Wizard.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- On the Welcome to SharePoint Products page, click Next.
- On the Connect to a server farm page, click Connect to an existing server farm.
- Click Next.
- On the Specify Configuration Database settings page, type the name of the instance of SQL Server in the Database server box, and then click Retrieve Database Names.
- Select the name of the configuration database in the Database name list, and then click Next.
- On the Specify Farm Security Settings page, type the name of the farm passphrase in the Passphrase box, and then click Next.
- On the Completing the SharePoint Products Configuration Wizard page, click Next.
- On the server that hosts Central Administration, click Manage servers in this farm to verify that the new server is part of the farm.
Note:
You can also verify a successful server addition or troubleshoot a failed addition by examining the log files. These files are located on the drive on which SharePoint 2013 is installed, in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder. For more information, see Monitor health in SharePoint 2013.
- On the Servers in Farm page, click the name of the new server. Use the list of available services on the Services on Server page to start the services that you want to run on the new server.
- Configure SharePoint 2013 so that the new server can accommodate the role for which it was intended. For more information, see Configure the new server.
To add a new SharePoint 2013 server to the farm by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:Right-click
-
Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command to connect the server to a configuration database:
Connect-SPConfigurationDatabase -DatabaseServer “<$DatabaseServer>” -DatabaseName “<$RunSettings.ConfigurationDatabaseName>” -Passphrase “<$Passphrase>“
Where:
- <$DatabaseServer> is the name of the server that hosts the configuration database
- <$RunSettings.ConfigurationDatabaseName> is the name of the configuration database
- <$Passphrase> is the passphrase for the farm
-
At the Windows PowerShell command prompt, type the following command to install the Help File Collections:
Install-SPHelpCollection -All
-
At the Windows PowerShell command prompt, type the following command to install the Security Resource for SharePoint 2013:
Initialize-SPResourceSecurity
-
At the Windows PowerShell command prompt, type the following command to install the basic services:
Install-SPService
-
At the Windows PowerShell command prompt, type the following command to install all the features:
Install-SPFeature -AllExistingFeatures
-
At the Windows PowerShell command prompt, type the following command to install application content:
Install-SPApplicationContent
-
At the Windows PowerShell command prompt, type the following command to get a list of servers in the farm.
Get-SPFarm | select Servers
Note:
You can also verify a successful server addition or troubleshoot a failed addition by examining the log files. These files are located on the drive on which SharePoint 2013 is installed, in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder. For more information, see Monitor health in SharePoint 2013.
- Configure SharePoint 2013 so that the new server can accommodate the role for which it was intended. For more information, see Configure the new server.
-
Configure the new server
The new server has no real functionality in the farm until you configure the services that are required to support the role that you planned for the new server. For more information, see Configure services and service applications in SharePoint 2013.
-
Remove a server from a farm in SharePoint 2013
Published: July 16, 2012
Summary: Learn how to remove a web server, application server, or database server from a SharePoint 2013 farm.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
There are three types of servers in a server farm running SharePoint 2013: web servers, application servers, and database servers. The method that you use to remove a server from a SharePoint farm varies depending on the type of server that you are removing from the farm.
In this article:
- Removing a web server or application server from a SharePoint farm
- Removing a database server from a SharePoint farm
- Remove a database server, web server, or application server from a SharePoint farm by using Central Administration
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Removing a web server or application server from a SharePoint farm
-
Removing a web server or application server from a SharePoint farm
For information about uninstallation procedures that SharePoint 2013 supports, see Uninstall SharePoint 2013.
Removing a server that contains a search topology component can affect future search activities. The extent of that effect depends on the farm search topology. We recommend that you remove or relocate any search topology components from a server before removing the server from the farm.
If you remove a server that hosts a crawl component, no index files are lost. However, you might reduce or remove the capacity to crawl content.
You can lose index files in the following situations:
- The farm has only one query component, and you remove the server that hosts the query component.
- You have configured the index to be partitioned and you delete the last query component in one of the partitions. In this case, you will lose a portion of the index.
In either of these cases, a full crawl will have to be performed to re-create the index files.
You can deploy specific techniques to build fault tolerance into the search topology. If these techniques are followed, the deliberate or unplanned removal of a server from the topology can be absorbed without losing data and without affecting the ability to crawl or serve queries. (However, performance can still be affected.) For more information, see Technical diagrams (SharePoint 2013).
Make sure that the server that you want to remove is not running any important site components. If important services or components (such as a custom Web Part) are running on the server and are not available on another server in the farm, removing the server can damage sites in the farm. For example, if the server that you want to remove is the only application server in the farm that is running the Business Data Connectivity service, removing the server can make any sites that rely on that service stop working correctly.
You can remove a web server or an application server from the server farm by uninstalling SharePoint 2013 from the server through Control Panel. When you uninstall SharePoint 2013 by using Control Panel, you remove the program files and other information from the server.
To remove a web server or an application server from a farm by using Control Panel
- The farm has only one query component, and you remove the server that hosts the query component.
- Verify that the user account that completes this procedure has the following credentials:
- The user account that performs this procedure is a member of the Administrators group on the server.
- Stop the services that are running on the server. For information about how to determine which services are running on a specific server and stopping services, see Start or Stop a service (SharePoint 2013).
- On the server that you want to remove from the farm, click Start, click Control Panel, and then double-click Programs and Features.
- In the list of currently installed programs, click SharePoint 2013, and then click Uninstall.
- Click Continue at the confirmation prompt to uninstall the program.
-
Removing a database server from a SharePoint farm
To remove a database server from a farm without uninstalling SharePoint and therefore deleting the data that was stored in the database, you must first move any databases that are hosted by that server to another database server in the farm and then use Central Administration to remove the database server from the farm.
You cannot remove a database server if it is the only database server available in the farm, or if it is the database server that hosts the configuration database.
Caution:
If you uninstall SharePoint 2013 from the server that is running Central Administration, you will be unable to administer the server farm until you configure another server in the farm to host the Central Administration site.
-
Remove a database server, web server, or application server from a SharePoint farm by using Central Administration
If a web server or application server is no longer available, or if uninstalling SharePoint 2013 from Control Panel is not possible, you can remove the web server or application server from the farm by using the SharePoint Central Administration website. Removing a server from the farm by using Central Administration does not uninstall SharePoint 2013 from the server, nor does it make any sites on that server inaccessible. We recommend that you use the process described in Remove a web server or an application server from a farm by using Control Panel to uninstall SharePoint 2013 instead of using Central Administration to remove the server.
Removing the server from the farm by using Central Administration does not delete this information from the server. Use the Central Administration procedure for removing database servers only, or for removing a web server or an application server from the farm when the server is no longer available to uninstall through Control Panel.
You can follow these steps to remove a web server, application server, or database server from the farm. However, we recommend that you remove web servers and application servers from a farm by using Control Panel, instead of by using Central Administration. For information, see To remove a web server or an application server from a farm by using Control Panel.
Before you remove a database server from a farm, make sure that you have moved any databases stored on that server to a different database server in your farm.
To remove a database server, web server, or application server from a SharePoint farm by using Central Administration
- Verify that the user account that completes this procedure has the following credentials:
- The user account that performs this procedure is a member of the Farm Administrators SharePoint group.
- The user account that performs this procedure is a member of the Administrators group on the server.
- Stop the services that are running on the server. For information about how to determine which services are running on a specific server and stopping services, see Start or Stop a service (SharePoint 2013).
- On the SharePoint Central Administration website, in the System Settings section, click Manage servers in this farm.
- On the Servers in Farm page, locate the row that contains the name of the server that you want to remove, and then click Remove Server.
-
In the warning that appears, click OK to remove the server or click Cancel to stop the operation.
The page updates, and the server that you removed no longer appears in the list of servers.
-
Uninstall SharePoint 2013
Published: July 16, 2012
Summary: SharePoint Server 2013 and SharePoint Foundation 2013 support a limited set of methods to uninstall.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 .
You remove SharePoint 2013 by uninstalling it from Control Panel. When you uninstall SharePoint 2013, most files and subfolders in the installation folders are removed. However, some files are not removed. Also,
- Web.config files, index files, log files, and customizations that you might have are not automatically removed when you uninstall SharePoint 2013.
- SQL Server databases are detached but are not removed from the database server.
- If you uninstall a single server that has a built-in database, SQL Server Express is not removed.
- When you uninstall SharePoint 2013, all user data remains in the database files.
- Web.config files, index files, log files, and customizations that you might have are not automatically removed when you uninstall SharePoint 2013.
-
Before you begin
Before you begin this operation, confirm that you have uninstalled all language packs that are on the server.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Uninstall SharePoint 2013
Use this procedure to uninstall SharePoint 2013.
To uninstall SharePoint 2013
- Verify that you are a member of the Farm Administrators group or a member of the Administrators group on the local computer.
- On the computer that runs SharePoint 2013, log on as a local or domain administrator.
- Start Control Panel.
-
For Windows Server 2008 R2:
- Click Start, and then click Control Panel.
- Click Start, and then click Control Panel.
-
For Windows Server 2012:
-
On the Start screen, click Control Panel.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- In the Programs area, click Uninstall a program.
- In the Uninstall or change a program dialog box, click Microsoft SharePoint Server 2013.
- Click Change.
-
On the Change your installation of Microsoft SharePoint Server 2013 page, click Remove, and then click Continue.
A confirmation message appears.
-
Click Yes to remove SharePoint 2013.
A warning message appears.
-
Click OK to continue.
A confirmation message appears.
-
Click OK.
You might be prompted to restart the server.
Note:
If you did not remove the language template packs before you uninstalled and then reinstalled SharePoint 2013, you must run Repair from the SharePoint Products Configuration Wizard for each language template pack on the server. After the repair operation is complete, you must restart the server. Finally, complete the language template pack configuration by running the SharePoint Products Configuration Wizard.
-
Install and configure a virtual environment for SharePoint 2013
Published: July 16, 2012
Summary: Learn about permissions, accounts, security settings, and what you have to do to prepare your Windows Server 2008 Hyper-V environment for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following downloadable resources, articles on TechNet, and related resources provide information about how to prepare Hyper-V to support a SharePoint 2013 virtual farm.
-
TechNet articles about SharePoint 2013 virtualization with Hyper-V
The following articles about virtualization in SharePoint 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Provides
Use best practice configurations for the SharePoint 2013 virtual machines and Hyper-V environment
Provides best practice guidance to install and configure virtual machines for a SharePoint 2013 farm and the Hyper-V virtualization host computer.
-
Additional resources about Hyper-V installation and initial configuration
The following resources about Hyper-V are available from other subject matter experts.
Content
Description
Visit the TechCenter to access videos, community sites, documentation, and more.
This TechNet Magazine article provides an overview of the preparation required to shift applications to a virtual environment.
-
Use best practice configurations for the SharePoint 2013 virtual machines and Hyper-V environment
Published: July 16, 2012
Summary: Follow best practice recommendations to configure the SharePoint 2013 virtual machines and the Windows Server 2008 Hyper-V infrastructure.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
After you create a detailed architecture design and system specifications, you are ready to install and configure the virtual environment for the SharePoint 2013 farm. To achieve the performance and capacity goals that you identify in your detailed design and system specifications, you must have a correctly configured virtual environment. One or more badly configured virtual machines or virtualization hosts can significantly reduce performance.
This article discusses Windows Server 2008 Hyper-V technology configuration options and their potential impact on the performance of the virtualization host computer, the virtual machines, and the SharePoint 2013 farm in general. Key decision points and best practice configurations provide guidance when you set up the virtual environment.
Important:
A wide range of features and installation options are available to set up and configure a Hyper-V environment. Measurements focus on testing and guidance on the features and configurations that provide quantifiable benefits and consider the known performance characteristics of server roles in a SharePoint products farm.
In this article:
- Introduction and scope
- Review the general best practice guidance for virtualization
- Configure the Hyper-V host computer
- Install and configure virtual networking
- Create and configure the virtual machines
- Configure the memory for the virtual machines
- Configure the processors for the virtual machines
- Configure the controllers and hard disks for the virtual machines
- Configure services and general settings
- Introduction and scope
-
Introduction and scope
We recommend that you use a bottom up approach to configure the Hyper-V environment for the SharePoint 2013 farm. Start with the Hyper-V host computer configuration and then work up to the virtual layer to configure the virtual network components and the virtual machines.
Before you install and configure the Hyper-V environment, we recommend that you review the following articles:
- Hardware Considerations for Hyper-V in Windows Server 2008
- Requirements and Limits for Virtual Machines and Hyper-V in Windows Server 2008 R2
-
-
Article scope
-
Several aspects about how to configure a virtual environment that are not discussed in detail are documented in separate articles. Detailed information about the following subjects is out of scope for this article:
- Business Continuity Management (high availability and disaster recovery) for a virtual environment
- Securing a virtual environment (options and best practices)
Note:
SharePoint 2013does not support the virtualization changes and features in Windows Server 2012.
- Hardware Considerations for Hyper-V in Windows Server 2008
-
Review the general best practice guidance for virtualization
When you install and configure virtualization for a SharePoint 2013 farm, every configuration choice that you make and implement has an effect on all other parts of the virtual environment. Your primary goal is to create intended consequences and positive effects. To achieve this goal you have to understand the interaction between the virtual and the physical, the inherent constraints. You also have to follow best practice guidance when you set up the virtual farm.
The first thing that you have to do before you install and configure your virtual environment is to make sure that you install the latest version of the Hyper-V Best Practices Analyzer (BPA). You can get this update from Update for Best Practices Analyzer for HYPER-V for Windows Server 2008 R2 x64 Edition (KB977238).
You can use Hyper-V Best Practices Analyzer to scan a server that is running the Hyper-V role and help identify configurations that do not comply with the best practices of Microsoft for this role. BPA scans the configuration of the physical computer, the virtual machines, and other resources, such as virtual networking and virtual storage. No configuration changes are made by running the scan.
The scan results display the following information:
- A list of issues that you can sort by severity
- Recommended fixes for issues
- Links to instructions
The best practices in the following table explain several aspects of virtualization and are not necessarily specific to any virtualization technology.
Note:
Hyper-V partition terminology
Hyper-V provides three types of partitions: the root partition, the parent partition, and the child partition. Each partition has its resources (memory and processor) and policies for device use. The root partition is the original partition and it starts the hypervisor. The parent partition calls the hypervisor to request the creation of new child partitions—the virtual machines.
-
Best practice guidance for virtualization
Best practice
Description
Leave adequate memory for the Hyper-V partitions.
Calculate the total memory requirements for all the virtual machines on the host and ensure that there is enough available memory to meet Hyper-V partition requirements.
Important:
For SharePoint products virtual machines, we recommend 4 GB of RAM or more for host computer operations.
Do not use the parent partition for services other than Hyper-V.
Do not run additional roles or services on the parent partition. Run them on the virtual machines instead because the parent is differentiated for scheduling.
Do not store host computer system files on drives that are used for Hyper-V storage.
To reduce disk contention, do not store system files (for example, Pagefile.sys) on hard disks that are dedicated to storing virtual machine data.
Use a minimum of two physical network adapters.
For better network management and performance, dedicate one adapter to virtual machine network traffic and use the other adapter for virtualization host network traffic.
Do not oversubscribe the CPU on the virtualization host computer.
Review the supported ratio of virtual processors per logical processor and avoid oversubscribing the host computer CPU. The optimum virtual processor:logical processor ratio is 1:1. For more information, see Configure the processors for the virtual machines.
Do not cross Non-uniform memory access (NUMA) boundaries.
Hyper-V spans NUMA nodes to assign physical memory to a virtual machine; however, this does reduce performance on the virtual machine. For more information, see Configure the memory for the virtual machines
Do not use snapshots in a production environment.
Do not use snapshots for the virtual machines in a SharePoint products production environment. When you create a snapshot, Hyper-V creates a new secondary drive for the virtual machine. Write operations occur on the new drive and read operations occur on both drives, which has the same net affect as a differencing disk. Every snapshot that you add reduces disk performance further.
- A list of issues that you can sort by severity
-
Configure the Hyper-V host computer
When you configure the Hyper-V host computer, we recommend that you consider the following configuration options and best practices:
-
Close the operating system management windows.
Hyper-V Manager and Virtual Machine Connection sessions consume resources. For example, Hyper-V manager causes Windows Management Instrumentation (WMI) activity in the parent partition. When you close the Virtual Machine Connection Manager, video emulation is disabled, which eliminate this source of resource consumption.
-
Avoid running programs such as anti-virus software on the parent partition.
Run them on the child partition if they are required.
-
Use hardware that supports Second Level Address Translation (SLAT).
SLAT is hardware that is optimized for virtualization improves virtual machine performance, and reduces processing load on the Windows hypervisor. For more information, see Hyper-V: List of SLAT-Capable CPUs for Hosts.
-
If your hardware supports Hyper-Threading, enable it.
Hyper-Threading splits the CPU pipeline in two and makes a single core look like two cores.
-
-
Install and configure virtual networking
Hyper-V provides the level of networking robustness and configuration options that you expect to see in a physical networking environment. There are, of course, some limitations because you are dealing with virtual devices.
We recommend that you refer to the following documentation to prepare to install and configure virtual networking.
- Understanding Networking with Hyper-V
- How does basic networking work in Hyper-V?
- Hyper-V: What are the uses for different types of virtual networks?
- Understanding Hyper-V VLANs
- Hyper-V VLANs Part II
-
-
Hyper-V virtual networks
-
You can create and configure a Hyper-V virtual network before you install and configure virtual machines. In addition, you can create more than one virtual network on a Hyper-V host computer.
Note:
You cannot create more than one virtual network on a Hyper-V host computer if the computer is running a Server Core installation of Windows Server 2008 or Windows Server 2008 R2.
Hyper-V provides three types of virtual networks that you can configure for virtual machines. The following table provides a summary of these virtual networks and their characteristics.
-
Virtual network types
Type
Description
External
Provides a communication link between virtual machines and a physical network by creating an association to a physical network adapter on the host computer. Dedicate one physical adapter to this type of network.
For security purposes you can isolate traffic between virtual machines and other computers on the physical network by clearing the Allow management operating system to share this network adapter setting. However, you will be unable to connect to the management operating system remotely.
Internal
Provides a communications link between the host computer and the virtual machines. This provides a degree of isolation from external network traffic and is typically used in a test environment where you want to connect to the virtual machines by using the management operating system.
Private
Provides a communications link between the virtual machines. They are completely isolated from the host computer and this type of virtual network is often used to set isolated test domains.
Important:
In a scenario where two internal (or private) virtual networks are created in Hyper-V and two virtual machines are created on a separate IP subnet, these virtual machines cannot communicate with one another. Because the virtual switch operates at layer 2 of the ISO/OSI Network Model, you have to have a router to achieve routing at a higher level.
After you create your virtual network, you can specify the range of media access control (MAC) addresses that are automatically assigned to the virtual network adapters. Hyper-V enables you to provide static MAC addresses to a virtual adapter to avoid collisions on the network.
Tip:
John Howard’s blog post, Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause provides a very good explanation about MAC Address allocation and associated network issues.
Windows Server 2008 R2 adds the option to configure MAC address spoofing (Enable Spoofing Of MAC Addresses) in the virtual network adapter settings. For more information, see Configure MAC Address Spoofing for Virtual Network Adapters.
-
Virtual local area networks (VLANs)
From a performance perspective, the ability to create virtual local area networks (VLANs) can provide significant throughput gains. Because virtual machines on the same VLAN can communicate through the virtual switch, network traffic is faster because it does not have to use the physical network adapter. Another benefit of a VLAN configuration is the fact that, because it is software-based, a virtual machine can easily be moved between hosts and still keeps its network configuration.
When you enable virtual LAN identification for the management operating system, you can assign a VLAN identifier (ID), which is an integer that uniquely identifies a node that belongs to a particular VLAN. If you use virtual LAN feature and a VLAN ID, note the following:
- The physical adapters must support VLAN tagging and this feature has to be enabled.
- Set the VLAN ID on either the virtual switch or on the virtual machine’s network adapter instead of the physical adapter.
-
You can assign only one VLAN ID on the virtual switch.
-
Network adapters and virtual network switches
-
Hyper-V provides two kinds of virtual network adapters that you can configure for a virtual machine: a network adapter and a legacy network adapter.
- A network adapter, also known as a synthetic adapter, is the preferred option for most virtual machine configurations. The driver for this adapter is included with the integration services that are installed with the Windows Server 2008 R2 guest operating system.
- A legacy adapter emulates an Intel 21140-based PCI Fast Ethernet Adapter, which results in a lower data transfer than the network adapter. A legacy network adapter also supports network-based installations because it can boot to the Pre-Boot Execution Environment (PXE).
Unless you have to use a legacy adapter until you can install the virtual machine driver or have to do a network boot, we recommend that you configure a virtual machine with a network adapter. If you do have to use a legacy adapter for a network installation, you can always add a network adapter later, and then delete the legacy adapter.
NIC teaming is the process of grouping several physical NICs into one logical NIC, which can be used for network fault tolerance and transmit load balance. The process of grouping NICs is called teaming. Teaming has two purposes:
- Fault tolerance. Teaming more than one physical NIC to a logical NIC maximizes high availability. Even if one NIC fails, the network connection does not stop and continues to operate on other NICs.
- Load balancing. Balancing the network traffic load on a server can enhance the functionality of the server and the network. Load balancing within network interconnect controller (NIC) teams enables distributing traffic among the members of a NIC team so that traffic is routed among all available paths.
Note:
Windows Server 2008 Service Pack 2 (SP2) and Windows Server 2008 R2 have no restrictions that are associated with NIC Teaming and the Failover Clustering feature. In Windows Server 2008, the Microsoft Failover Cluster Virtual Adapter is compatible with NIC Teaming and enables it to be used on any network interface in a Failover Cluster.
Windows Server 2008 R2 adds important new capabilities that you should use in your Hyper-V environment if your servers and network hardware support them. We recommend that you investigate the following networking options:
- Large Send Offload (LSO) and Checksum Offload (CSO). The virtual networks in Hyper-V support LSO and CSO. In addition, if your physical network adapters support these capabilities, the virtual traffic is offloaded to the physical network as necessary. Most network adapters support LSO and CSO.
- Jumbo frames. With Windows Server 2008 R2, jumbo frame improvements converge to support up to 6 times the payload per packet. This increases overall throughput and reduces CPU utilization for large file transfers. Physical networks and virtual networks support jumbo frames. This includes switches and adapters.
Note:
For physical networks, all intervening network hardware such as switches must have jumbo frame support enabled also.
- TCP chimney. This lets virtual NICs in child partitions to offload TCP connections to physical adapters that support it, which reduces CPU utilization and other overhead.
- Virtual machine queue (VMQ). VMQ improves network throughput by distributing network traffic for multiple VMs across multiple processors. This process reduces processor utilization by offloading packet classification to the hardware and avoiding both network data copy and route lookup on transmit paths. VMQ is compatible with most other task offloads and can coexist with large send offload and jumbo frames.
- Understanding Networking with Hyper-V
-
Create and configure the virtual machines
Although Hyper-V supports several guest operating systems, SharePoint 2013 requires the 64-bit edition of Windows Server 2008 R2 Service Pack 1 (SP1) Standard, Enterprise, or Data Center. For more information about supported guest operating systems, see About Virtual Machines and Guest Operating Systems
Hyper-V provides many configuration options and you can change a configuration, the amount of memory for example, after the virtual machine is running as a SharePoint products farm server. With the exception of adding a virtual hard disk drive for a SCSI controller, you have to shut down a virtual machine before you can change its configuration.
Configure each virtual machine according to the requirements in Capacity management and high availability in a virtual environment (SharePoint Server 2010). Configure the following for each virtual machine:
- The BIOS setting to set the boot sequence (legacy network adapter, CD, IDE, or floppy disk)
- The amount of memory
- The number of virtual processors
- The type and number of controllers
- The type and number of hard disks
- The type and number of network adapters
In addition to the previous configurations, you also have the option to configure a DVD drive, COM ports, and a virtual floppy disk.
From a SharePoint products perspective, the primary configuration considerations are the memory, processor, and hard disks.
- The BIOS setting to set the boot sequence (legacy network adapter, CD, IDE, or floppy disk)
-
Configure the memory for the virtual machines
Configure memory on a virtual machine as you typically do for an application that runs on a physical server. The memory allocation must be sufficient to reasonably handle the load at ordinary and peak times. For SharePoint virtual machines, insufficient memory is the main cause of performance issues.
Before you install and configure the virtual machines on a Hyper-V host computer, calculate how much memory is available for the virtual machines. The root partition must have sufficient memory to provide services such as I/O virtualization and management to support the child partitions. For SharePoint products, we recommend that you allow a minimum of 4 GB of RAM for overhead on a Hyper-V virtualization host computer.
After you factor in the 4 GB RAM reserve for the virtualization host, configure the virtual machines to use the remaining memory.
Dynamic memory
Windows Server 2008 R2 SP1 has the option of configuring dynamic memory (with a minimum value and maximum value) for virtual machines.
We do not support this option for virtual machines that run in a SharePoint 2013 environment. The reason is that this implementation of dynamic memory does not work with every SharePoint feature. For example, Distributed Cache and Search do not resize their caches when the allocated memory for a virtual machine is dynamically changed. This can cause performance degradation, especially when assigned memory is reduced.
Non-uniform memory access (NUMA)
A very import aspect of virtual machine memory configuration is Non-uniform memory access (NUMA). NUMA is a memory design that speeds up memory access by partitioning physical memory so each processor in a multi-CPU has its own memory. For example, in a system with 8 cores and 32 GB of RAM, each core or node has 4 GB of physical memory. If a virtual machine is configured to use 8 GB of RAM, the system has to use memory in another node. Because crossing the NUMA boundary can reduce virtual performance by as much as 8%, it is a best practice to configure a virtual machine to use resources from a single NUMA node. For more information about NUMA, refer to the following articles:
-
Configure the processors for the virtual machines
You can configure multiple virtual processors for a virtual machine, up to a limit of four processors. You cannot configure more processors per virtual machine than there are logical (cores) processors on the virtualization host. For example, given a dual core physical server, the limit is two virtual processors for a virtual machine. Although Hyper-V supports up to eight virtual processors per core or per thread, a configuration that exceeds this ratio (8:1) is known as being oversubscribed. For any virtual machine that you use in a SharePoint 2013 farm, we recommend a ratio of 1:1. Oversubscribing the CPU on the virtualization host can decrease performance, depending on how much the CPU is oversubscribed. For more information, see Hyper-V VM Density, VP:LP Ratio, Cores and Threads
-
Configure the controllers and hard disks for the virtual machines
You can use two controller options and several hard disk configurations for a virtual machine. Before you configure storage for your virtual machines, read the following posts written by Jose Barreto, who is a member of the File Server Team at Microsoft.
- Storage options for Windows Server 2008 Hyper-V
- More on Storage Options for Windows Server 2008 Hyper-V
You can select either Integrated Device Electronics or SCSI devices on virtual machines, as follows:
- IDE devices: Hyper-V uses emulated devices with IDE controllers. You can have up to two IDE controllers with two disks on each controller. The startup disk (also known as the boot disk) must be attached to one of the IDE devices. The startup disk can be either a virtual hard disk or a physical disk. Although a virtual machine must use an IDE device as the startup disk to start the guest operating system, you have many options to choose from when you select the physical device that will provide the storage for the IDE device.
- SCSI devices: Each virtual machine supports up to 256 SCSI disks (four SCSI controllers with each controller supporting up to 64 disks). SCSI controllers use a type of device that was developed specifically for use with virtual machines and use the virtual machine bus to communicate. The virtual machine bus must be available when the guest operating system is started. Therefore, virtual hard disks that are attached to SCSI controllers cannot be used as startup disks.
Note:
Physical SCSI devices typically provide better I/O performance than physical IDE devices. However, this is not the case for virtualized SCSI and IDE devices in Hyper-V. Support for hot swappable hard disk drives, which the Hyper-V implementation of SCSI supports, is a better reason for selecting SCSI drives than performance gains.
The version of Hyper-V released with Windows Server 2008 R2 provides significant improvements in virtual hard disk performance. For more information, see Virtual Hard Disk Performance: Windows Server 2008 / Windows Server 2008 R2 / Windows 7. For a summary of virtual machine drive options, see the “How to choose your Hyper-V and VHD Storage Container Format” section of ” Virtual Hard Disk Performance: Windows Server 2008 / Windows Server 2008 R2 / Windows 7″.
Hyper-V supports many storage options. For more information about the storage options, see Planning for Disks and Storage.
You can use the following types of physical storage with a server that runs Hyper-V:
- Direct-attached storage: You can use Serial Advanced Technology Attachment (SATA), external Serial Advanced Technology Attachment (eSATA), Parallel Advanced Technology Attachment (PATA), Serial Attached SCSI (SAS), SCSI, USB, and Firewire.
- Storage area networks (SANs): You can use Internet SCSI (iSCSI), Fibre Channel, and SAS technologies.
For more information, see Configuring Pass-through Disks in Hyper-V.
There is no generic storage solution for every virtual environment. Selecting the optimal virtual machine drive option for your SharePoint 2013 servers requires research and extensive testing to implement the best storage solution for your virtual environment. When you pick a storage solution you must consider access performance, storage needs, and how much memory is used for the advanced caching of virtual hard disk images.
- Storage options for Windows Server 2008 Hyper-V
-
Configure services and general settings
In a Hyper-V environment, you can specify the configuration of virtual networking and the configuration for each virtual machine. Additionally, you can configure how each virtual machine interacts with the Hyper-V host computer, and also the stop and restart behavior if the running state of the virtual machine is interrupted.
-
Integration services
Hyper-V includes a software package for supported guest operating systems that improves integration between the physical computer and the virtual machine. This package is known as integration services. You should verify that the management operating system (which runs the Hyper-V role) and virtual machines are running the same version of integration services. For more information, see Version Compatibility for Integration Services.
For each virtual machine you can configure the following integration items between the virtual machine and the virtualization host computer:
- Operating system shutdown
- Time synchronization
- Data exchange
- Heartbeat
- Backup (volume snapshot)
Important:
Disable the time synchronization for each SharePoint virtual machine. SharePoint 2013 implements timer jobs extensively and the latency during time synchronization will cause unpredictable results in the SharePoint environment.
-
Automatic stop and start
For each virtual machine you can configure automatic stop and start behavior if a physical computer shuts down.
The options for stop are as follows:
-
Save the virtual machine state.
The current state of the virtual machine is saved. When the virtual machine is started, Hyper-V attempts to restore the virtual machine to the state it was in.
-
Turn off the virtual machine.
This is the equivalent of pulling the power plug on a server.
-
Shut down the guest (virtual machine) operating system.
This is the equivalent of shutting down a computer by using the Windows Shut down option.
For a SharePoint products virtual machine, do not configure the virtual machine to save state. Virtual machines that start from saved state will be out of synchronization with the other servers in the farm. We recommend that you configure the virtual machine to use a shutdown because it minimizes that chances that the virtual machine can be corrupted. When a shutdown happens, all timer jobs that are running can finish, and there will be no synchronization issues when the virtual machine restarts.
The opposite of an automatic stop is an automatic start. Hyper-V provides the following startup options when the physical server restarts:
-
Do nothing.
You have to start the virtual machine manually regardless of its state when the physical server shut down.
- Automatically start if the virtual machine was running when the service stopped.
-
Always start this virtual machine automatically.
Hyper-V starts the virtual machine regardless of its state when the physical server shut down.
We recommend that you select either of the first two options. Both options are acceptable. However, the decision is ultimately up to the IT team that manages and maintains the virtual environment.
In addition to the previous start options, you can configure a startup time delay for a virtual machine. We recommend that you do this to reduce resource contention on a virtualization host. However, if your start option is to do nothing, this is not an issue.
-
-
Configure SharePoint 2013
Published: July 16, 2012
Summary: Lists articles that describe how to configure settings (such as services, authentication, and specific features) after you install SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
After you install SharePoint 2013, you must configure several additional settings to enable key features in your farm. The articles in this section provide steps for configuring these settings.
-
TechNet articles about how to configure settings for the server farm
The following articles about how to configure settings for the server farm are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
These articles describe how to configure the infrastructure for user, server-to-server, and app authentication.
Configure availability and recovery solutions for SharePoint 2013
Learn about the options for providing high availability and disaster recovery solutions for SharePoint 2013.
These articles describe how to configure incoming and outgoing e-mail in the server farm.
Learn about new licensing functionality and how to configure licensing in SharePoint Server 2013.
Learn how to subscribe to SMS alerts for your mobile device in SharePoint Server 2013.
Learn how Request Manager can route and throttle incoming requests to help improve performance and availability.
Configure services and service applications in SharePoint 2013
These articles describe how to configure services and service applications.
Configure usage and health data collection (SharePoint 2013)
Learn how to configure usage and health data collection.
Configure Business Connectivity Services solutions for SharePoint 2013
Find links to procedures to help you install and configure Business Connectivity Services for SharePoint 2013 on-premises and other scenarios.
Learn the steps to set up and configure eDiscovery in SharePoint Server 2013 and Exchange Server 2013.
Configure Exchange task synchronization in SharePoint Server 2013
Configure Exchange Server 2013 and SharePoint Server 2013 for task synchronization by using the SharePoint Server 2013 Task Synchronization feature.
Configure Exchange Server 2013 and SharePoint Server 2013 for team email by using the SharePoint Server 2013 Site Mailboxes feature.
Configure social computing features in SharePoint Server 2013
These articles describe how to configure social computing features. This includes My Sites, Community Sites, and microblogging in SharePoint Server 2013.
Configure web content management solutions in SharePoint Server 2013
Learn how to configure SharePoint web content management solutions that use cross-site collection publishing in SharePoint Server 2013.
Learn how to install and configure Workflow Manager in SharePoint Server 2013.
-
Additional resources about how to configure settings for the server farm
The following resources about how to configure settings for the server farm are available from other subject matter experts.
Content
Description
Installation and Deployment for SharePoint 2013 Resource Center
Visit the Resource Center to access videos, community sites, documentation, and more.
-
Configure authentication infrastructure in SharePoint 2013
Published: July 16, 2012
Summary: Find resources to help you configure the infrastructure for user, server-to-server, and app authentication in SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following articles on TechNet and related resources provide information about how to configure authentication infrastructure.
-
TechNet articles about how to configure authentication infrastructure
The following articles about how to configure authentication infrastructure in SharePoint 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Configure forms-based authentication for a claims-based web application in SharePoint 2013
Describes the steps to configure forms-based authentication using a Lightweight Directory Access Protocol (LDAP) membership provider.
Configure SAML-based claims authentication with AD FS in SharePoint 2013
Describes the steps to configure Security Assertion Markup Language (SAML)-based claims authentication using Active Directory Federation Services (AD FS) 2.0.
Configure server-to-server authentication in SharePoint 2013
These articles describe the steps to configure server-to-server authentication.
Describes the steps to configure app authentication for SharePoint Server 2013.
Configure client certificate authentication for SharePoint 2013
Describes how to configure user authentication with client certificates.
-
Configure forms-based authentication for a claims-based web application in SharePoint 2013
Updated: October 2, 2012
Summary: Learn how to configure forms-based authentication with an LDAP provider for a new SharePoint 2013 web application.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article provides guidance for configuring forms-based authentication for a SharePoint 2013 web application that uses a Lightweight Directory Access Protocol (LDAP) membership provider. Forms-based authentication is an identity management system that is based on ASP.NET membership and role provider authentication. Forms-based authentication in SharePoint 2013 is a claims-based authentication method. For more information about the use of forms-based authentication, see the “Implementing forms-based authentication“ section of Plan for user authentication methods.
Important:
The steps in this article apply to SharePoint Server 2013.
For a version of these procedures that are configured in a standardized test lab, see Test Lab Guide: Demonstrate forms-based claims authentication for SharePoint Server 2013.
-
Before you begin
Before you begin this operation, you should be familiar with the concepts in Plan for user authentication methods.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Process overview
This configuration has the following phases that must be performed in consecutive order:
- Phase 1: Create a new web application that uses forms-based authentication with Central Administration
- Phase 2: Configure the Web.Config files for an LDAP membership provider
Within each phase, the set of procedures must also be performed in consecutive order.
For an alternative to creating the new web application by using Central Administration, see Create a new web application that uses forms-based authentication with Windows PowerShell.
If you are using iFrame-based Windows Azure autohosted apps within the web application, see Configure a forms-based authentication web application for Windows Azure autohosted apps.
- Phase 1: Create a new web application that uses forms-based authentication with Central Administration
-
Phase 1: Create a new web application that uses forms-based authentication with Central Administration
Perform the steps in the following procedure to create a web application that uses forms-based authentication with Central Administration.
To create a new web application that uses forms-based authentication with Central Administration
- Verify that the user account that is performing this procedure is a site collection administrator.
- Start SharePoint 2013 Central Administration.
-
For Windows Server 2008 R2:
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Central Administration.
If SharePoint 2013 Central Administration is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Central Administration.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- In Central Administration, in the Application Management section, click Manage web applications.
- In the Contribute group of the ribbon, click New.
- In the Claims Authentication Types section of the Create New Web Application dialog box, select Enable Forms Based Authentication (FBA).
-
Type a membership provider name in ASP.NET Membership provider name and a role manager name in ASP.NET Role manager name.
In the example Web.Config files depicted in this article, the membership provider is membership and the role manager is rolemanager.
- Configure the other settings for this new web application as needed, and then click OK to create it.
- When prompted with the Application Created dialog box, click OK.
-
Phase 2: Configure the Web.Config files for an LDAP membership provider
After you successfully create the new web application, modify the following Web.Config files in every web front-end server in the farm:
- To configure the Central Administration Web.Config file
- To configure the Security Token Service Web.Config file
-
-
Configure the Central Administration Web.Config file
-
The following procedure configures the Central Administration web site to recognize and use the new forms-based membership provider and role manager.
To configure the Central Administration Web.Config file
- To configure the Central Administration Web.Config file
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) Manager.
- In the console tree, open the server name, and then Sites.
- Right-click the SharePoint Central Administration v4 site, and then click Explore.
- In the folder window, double-click the Web.Config file.
-
In the <Configuration> section, find the <system.web> section and add the following example entry:
<membership defaultProvider=”AspNetSqlMembershipProvider”>
<providers>
<add name=”membership”
type=”Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
userDNAttribute=”distinguishedName”
userNameAttribute=”sAMAccountName”
userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=distinguishedName (of your userContainer)”
userObjectClass=”person”
userFilter=”(ObjectClass=person)”
scope=”Subtree”
otherRequiredUserAttributes=”sn,givenname,cn” />
</providers>
</membership>
<roleManager enabled=”true” defaultProvider=”AspNetWindowsTokenRoleProvider” >
<providers>
<add name=”roleManager”
type=”Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
groupContainer=”DC=internal,DC=yourcompany,DC=distinguishedName (of your groupContainer)”
groupNameAttribute=”cn”
groupNameAlternateSearchAttribute=”samAccountName”
groupMemberAttribute=”member”
userNameAttribute=”sAMAccountName”
dnAttribute=”distinguishedName”
groupFilter=”((ObjectClass=group)”
userFilter=”((ObjectClass=person)”
scope=”Subtree” />
</providers>
</roleManager>
In the preceding entry, substitute the following:
- The name of your membership provider in <add name=”membership”.
- The fully qualified domain name (FQDN) of your domain controller (your LDAP server) in server=”yourserver.com”.
- The distinguished name of your user container in userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=distinguishedName (of your userContainer)”.
- The name of your role manager in <add name=”roleManager”.
- The distinguished name of your group container in groupContainer=”DC=internal,DC=yourcompany,DC=distinguishedName (of your groupContainer)”.
After you add this entry, save and close the Web.Config file.
-
Configure the Security Token Service Web.Config file
The following procedure configures the Security Token Service to recognize and use the new forms-based membership provider and role manager.
To configure the Security Token Service Web.Config file
- In the console tree of Internet Information Services (IIS) Manager, open the SharePoint Web Services site.
- In the console tree, right-click SecurityTokenServiceApplication, and then click Explore.
- In the folder window, double-click the Web.Config file.
-
In the <Configuration> section, create a new <system.web> section and add the following example entry:
<membership>
<providers>
<add name=”membership”
type=”Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
userDNAttribute=”distinguishedName”
userNameAttribute=”sAMAccountName”
userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=com”
userObjectClass=”person”
userFilter=”(&(ObjectClass=person))”
scope=”Subtree”
otherRequiredUserAttributes=”sn,givenname,cn” />
</providers>
</membership>
<roleManager enabled=”true” >
<providers>
<add name=”rolemanager”
type=”Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
groupContainer=”DC=internal,DC=yourcompany,DC=com”
groupNameAttribute=”cn”
groupNameAlternateSearchAttribute=”samAccountName”
groupMemberAttribute=”member”
userNameAttribute=”sAMAccountName”
dnAttribute=”distinguishedName”
groupFilter=”(&(ObjectClass=group))”
userFilter=”(&(ObjectClass=person))”
scope=”Subtree” />
</providers>
</roleManager>
In the preceding entry, substitute the following:
- The name of your membership provider in <add name=”membership”.
- The FQDN of your domain controller (your LDAP server) in server=”yourserver.com”.
- The distinguished name of your user container in userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=com”.
- The name of your role manager in <add name=”roleManager”.
- The distinguished name of your group container in groupContainer=”DC=internal,DC=yourcompany,DC=com”.
After you add this entry, save and close the Web.Config file.
-
Configure the new web application Web.Config file
The following procedure configures the new web application to recognize and use the new forms-based membership provider and role manager.
To configure the new web application Web.Config file
- In the console tree of Internet Information Services (IIS) Manager, right-click the site that corresponds to the name of the web applications that you just created, and then click Explore.
- In the folder window, double-click the Web.Config file.
- In the <Configuration> section, find the <system.web> section.
-
Find the <membership defaultProvider=”i”> section and add the following example entry to the <Providers> section:
<add name=”membership”
type=”Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
userDNAttribute=”distinguishedName”
userNameAttribute=”sAMAccountName”
userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=com”
userObjectClass=”person”
userFilter=”(&(ObjectClass=person))”
scope=”Subtree”
otherRequiredUserAttributes=”sn,givenname,cn” />
In the preceding entry, substitute the following:
- The name of your membership provider in <add name=”membership”.
- The FQDN of your domain controller (your LDAP server) in server=”yourserver.com”.
- The distinguished name of your user container in userContainer=”OU=UserAccounts,DC=internal,DC=yourcompany,DC=com”.
- Find the <roleManager defaultProvider=”c” enabled=”true” cacheRolesInCookie=”false”> section and add the following example entry to the <Providers> section:
<add name=”roleManager”
type=”Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c”
server=”yourserver.com”
port=”389″
useSSL=”false”
groupContainer=”DC=internal,DC=yourcompany,DC=com”
groupNameAttribute=”cn”
groupNameAlternateSearchAttribute=”samAccountName”
groupMemberAttribute=”member”
userNameAttribute=”sAMAccountName”
dnAttribute=”distinguishedName”
groupFilter=”(&(ObjectClass=group))”
userFilter=”(&(ObjectClass=person))”
scope=”Subtree” />
In the preceding entry, substitute the following:
- The name of your role manager in <add name=”roleManager”.
- The FQDN of your domain controller (your LDAP server) in server=”yourserver.com”.
- The distinguished name of your group container in groupContainer=”DC=internal,DC=yourcompany,DC=com”.
After you add the preceding entry, save and close the Web.Config file.
Warning:
Do not overwrite any existing entries in this Web.Config file.
-
Create a new web application that uses forms-based authentication with Windows PowerShell
Perform the following procedure to create a web application that uses forms-based authentication with Windows PowerShell.
To create a new web application that uses forms-based authentication with Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
From the Windows PowerShell command prompt, type the following:
$ap = New-SPAuthenticationProvider -Name <Name> -ASPNETMembershipProvider <Membership Provider Name> -ASPNETRoleProviderName <Role Manager Name>
$wa = New-SPWebApplication -Name <Name> -ApplicationPool <ApplicationPool> -ApplicationPoolAccount <ApplicationPoolAccount> -Url <URL> -Port <Port> -AuthenticationProvider $ap
Example
$ap = New-SPAuthenticationProvider -Name “ClaimsForms” -ASPNETMembershipProvider “membership” -ASPNETRoleProviderName “rolemanager”
$wa = New-SPWebApplication -Name “FBA Web App” -ApplicationPool “Claims App Pool” -ApplicationPoolAccount “internal\appool” -Url http://contoso.com -Port 1234 -AuthenticationProvider $ap
Note:
The value of the ApplicationPoolAccount parameter must be a managed account on the farm.
- After you successfully create the new web application, modify the following Web.Config files:
- To configure the Central Administration Web.Config file
- To configure the Security Token Service Web.Config file
- To configure the new web application Web.Config file
-
After you change the Web.Config files, create a SPClaimsPrincipal and a site collection, as shown in the following example:
$cp = New-SPClaimsPrincipal -Identity “membership:SiteOwner” -IdentityType FormsUser
$sp = New-SPSite http://servername:port -OwnerAlias $cp.Encode() -Template “STS#0”
For more information, see New-SPClaimsPrincipal.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Configure a forms-based authentication web application for Windows Azure autohosted apps
To support iFrame-based Windows Azure autohosted apps from a SharePoint 2013 web application that is configured for forms-based authentication, you must complete the following procedure. For more information about apps for SharePoint, see Overview of apps for SharePoint 2013.
To configure a forms-based authentication web application to support Windows Azure autohosted apps
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
From the Windows PowerShell command prompt, type the following:
$svc = [Microsoft.SharePoint.Administration.SPWebService]::ContentService
$svc.MembershipUserKeyType=[Microsoft.SharePoint.Administration.SPMembershipUserKeyType]::ProviderUserKey
$svc.Update()
-
Configure SAML-based claims authentication with AD FS in SharePoint 2013
Updated: October 16, 2012
Summary: Learn how to configure Security Assertion Markup Language (SAML)-based claims authentication using Active Directory Federation Services version 2.0 (AD FS).
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The procedures in this article describe how to configure AD FS to act as an Identity Provider Security Token Service (IP-STS) for a SharePoint 2013 web application. In this configuration, AD FS issues SAML-based security tokens consisting of claims so that client computers can access web applications that use claims-based authentication. You can use an alternative identity provider than AD FS, but it must support the WS-Federation standard.
For information about why you would use SAML-based authentication, see Plan for user authentication methods.
You can use AD FS with the Windows Server 2012, Windows Server 2008, or Windows Server 2008 R2 operating systems to build a federated identity management solution that extends distributed identification, authentication, and authorization services to web-based applications across organization and platform boundaries. By deploying AD FS, you can extend your organization‘s existing identity management capabilities to the Internet.
For a version of these procedures that are configured in a standardized test lab, see Test Lab Guide: Demonstrate SAML-based Claims Authentication with SharePoint Server 2013.
-
Before you begin
Before you begin this operation, you should be familiar with the concepts in the following article:
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Process overview
This configuration has the following phases that must be performed in consecutive order:
- Phase 1: Install and configure an AD FS server
- Phase 2: Configure AD FS with the web application as a relying party
- Phase 3: Configure SharePoint 2013 to trust AD FS as an identity provider
- Phase 4: Configure web applications to use claims-based authentication and AD FS as the trusted identity provider
Within each phase, the set of procedures must also be performed in consecutive order.
- Phase 1: Install and configure an AD FS server
-
Phase 1: Install and configure an AD FS server
You must install and configure a server that runs AD FS 2.0. For more information, see the AD FS 2.0 Deployment Guide (http://go.microsoft.com/fwlink/p/?LinkId=191723).
-
Phase 2: Configure AD FS with the web application as a relying party
This phase has the following procedures:
Use the procedure in this section to configure a relying party. The relying party defines how the AD FS recognizes the relying party application and issues claims to it.
To configure AD FS for a relying party
- Verify that the user account that is performing this procedure is a member of the Administrators group on the local computer. For additional information about accounts and group memberships, see Local and Domain Default Groups
- On the AD FS server, open the Active Directory Federation Services (AD FS) 2.0 Management console.
- In the navigation pane, expand Trust Relationships, and then double-click the Relying Party Trusts folder.
-
In the right pane, click Add Relying Party Trust.
This opens the Active Directory Federation Services (AD FS) 2.0 configuration wizard.
- On the Welcome to the Add Relying Party Trust Wizard page, click Start.
- Select Enter data about the relying party manually, and then click next.
- Type a relying party name and then click Next.
- Make sure Active Directory Federation Services (AD FS) 2.0 Profile is selected, and then click Next.
- Do not use an encryption certificate. Click Next.
- Click to select the Enable support for the WS-Federation Passive protocol check box.
- In the WS-Federation Passive protocol URL field, type the name of the web application URL, and append /_trust/ (for example, https://WebAppName/_trust/). Click Next.
Note:
The name of the URL has to use Secure Sockets Layer (SSL).
- Type the name of the relying party trust identifier (for example, urn:sharepoint:WebAppName), and then click Add. Click Next. Note that this will be the realm value when you configure a new SPTrustedIdentityTokenIssuer in Phase 3.
- Select Permit all users to access this relying party. Click Next.
- On the Ready to Add Trust page, there is no action required, click Next.
- On the Finish page, click Close. This opens the Rules Editor Management console. Use this console and the next procedure to configure the mapping of claims from your chosen directory source to SharePoint 2013.
Use the procedure in this step to send values of a Lightweight Directory Access Protocol (LDAP) attribute as claims and specify how the attributes will map to the outgoing claim type.
To configure a claim rule
- Verify that the user account that is performing this procedure is a member of the Administrators group on the local computer. For additional information about accounts and group memberships, see Local and Domain Default Groups
- On the Issuance Transform Rules tab, click Add Rule.
- On the Select Rule Template page, select Send LDAP Attributes as Claims. Click Next.
- On the Configure Rule page, type the name of the claim rule in the Claim rule name field.
- From the Attribute Store drop-down list, select Active Directory.
- In the Mapping of LDAP attributes to outgoing claim types section, under LDAP Attribute, select SAM-Account-Name.
- Under Outgoing Claim Type, select E-Mail Address.
- Under LDAP Attribute, select User-Principal-Name.
- Under Outgoing Claim Type, select UPN.
- Click Finish, and then click OK.
Use the procedure in this section to export the token signing certificate of the AD FS server with which you want to establish a trust relationship, and then copy the certificate to a location that SharePoint 2013 can access.
To export a token signing certificate
- Verify that the user account that is performing this procedure is a member of the Administrators group on the local computer. For additional information about accounts and group memberships, see Local and Domain Default Groups
- On the AD FS server, open the Active Directory Federation Services (AD FS) 2.0 Management console.
- In the navigation pane, expand Service, and then click the Certificates folder.
- Under Token signing, click the primary token certificate as indicated in the Primary column.
- In the right pane, click View Certificate link. This displays the properties of the certificate.
- Click the Details tab.
- Click Copy to File. This starts the Certificate Export Wizard.
- On the Welcome to the Certificate Export Wizard page, click Next.
- On the Export Private Key page, click No, do not export the private key, and then click Next.
- On the Export File Format page, select DER encoded binary X.509 (.CER), and then click Next.
- On the File to Export page, type the name and location of the file that you want to export, and then click Next. For example, enter C:\ADFS.cer.
- On the Completing the Certificate Export Wizard page, click Finish.
-
Phase 3: Configure SharePoint 2013 to trust AD FS as an identity provider
This phase has the following procedures:
- Exporting multiple parent certificates
- Import a token signing certificate by using Windows PowerShell
- Define a unique identifier for claims mapping by using Windows PowerShell
- Create a new authentication provider
To complete the configuration of the AD FS server, copy the .CER file to the computer that is running AD FS.
The token signing certificate may have one or more parent certificates in its chain. If it does, every certificate in that chain has to be added to the SharePoint 2013 list of trusted root authorities.
To determine whether one or more parent certificates exist, follow these steps.
Note:
These steps should be repeated until all certificates are exported up to the root authority certificate.
To export multiple parent certificates
- Verify that the user account that is performing this procedure is a member of the Administrators group on the local computer. For additional information about accounts and group memberships, see Local and Domain Default Groups
- Open the Active Directory Federation Services (AD FS) 2.0 Management console.
- In the navigation pane, expand Service, and then click the Certificates folder.
- Under Token signing, click the primary token certificate as indicated in the Primary column.
- In the right pane, click View Certificate link. This displays the properties of the certificate.
- Click the Certification tab. This displays any other certificate(s) in the chain.
- Click the Details tab.
- Click Copy to File. This starts the Certificate Export Wizard.
- On the Welcome to the Certificate Export Wizard page, click Next.
- On the Export Private Key page, click No, do not export the private key, and then click Next.
- On the Export File Format page, select DER encoded binary X.509 (.CER), and then click Next.
- On the File to Export page, type the name and location of the file that you want to export, and then click Next. For example, enter C:\adfsParent.cer.
- On the Completing the Certificate Export Wizard page, click Finish.
Use this section to import the token signing certificates to the trusted root authority list that resides on the SharePoint Server. This step must be repeated for every token signing certificate in the chain until the root certification authority is reached.
To import a token signing certificate by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
From the Windows PowerShell command prompt, import the parent certificate of the token signing certificate (that is, the root authority certificate), as shown in the following syntax:
$root = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(“<PathToParentCert>“)
New-SPTrustedRootAuthority -Name “Token Signing Cert Parent” -Certificate $root
-
From the Windows PowerShell command prompt, import the token signing certificate that was copied from the AD FS server, as shown in the following syntax:
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(“<PathToSigningCert>“)
New-SPTrustedRootAuthority -Name “Token Signing Cert” -Certificate $cert
For additional information about the New-SPTrustedRootAuthority cmdlet, see New-SPTrustedRootAuthority
Use the procedure in this section to define a unique identifier for claims mapping. Typically, this information is in the form of an e-mail address and the administrator of the trusted STS will have to provide this information because only the owner of the STS knows which claim type will be always unique for each user.
To define a unique identifier for claims mapping by using Windows PowerShell
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
From the Windows PowerShell command prompt, create an identity claim mapping, as shown in the following syntax:
$emailClaimMap = New-SPClaimTypeMapping -IncomingClaimType “http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress” -IncomingClaimTypeDisplayName “EmailAddress” -SameAsIncoming
-
From the Windows PowerShell command prompt, create the UPN claim mapping as shown in the following syntax:
$upnClaimMap = New-SPClaimTypeMapping -IncomingClaimType “http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn” -IncomingClaimTypeDisplayName “UPN” -SameAsIncoming
For additional information about the New-SPClaimTypeMapping cmdlet, see New-SPClaimTypeMapping
Use the procedure in this section to create a new SPTrustedIdentityTokenIssuer.
To create a new authentication provider by using Windows PowerShell
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- From the Windows PowerShell command prompt, create a new authentication provider, as shown in the following syntax.
Note:
The $realm variable defines the trusted STS that identifies a specific SharePoint farm and the $cert variable is the one that was used from the Import a token signing certificate by using Windows PowerShell section. The SignInUrl parameter is to the AD FS server.
$realm = “urn:sharepoint:<WebAppName>“
$signInURL = “https://<YourADFSServerName>/adfs/ls”
$ap = New-SPTrustedIdentityTokenIssuer -Name <ProviderName> -Description <ProviderDescription> -realm $realm -ImportTrustCertificate $cert -ClaimsMappings $emailClaimMap,$upnClaimMap -SignInUrl $signInURL -IdentifierClaim $emailClaimmap.InputClaimType
For additional information about the New-SPTrustedIdentityTokenIssuer cmdlet, see New-SPTrustedIdentityTokenIssuer
-
Phase 4: Configure web applications to use claims-based authentication and AD FS as the trusted identity provider
This phase has the following procedures:
- Associate an existing web application with the AD FS identity provider
- Create a new web application with the AD FS identity provider
To configure an existing web application to use SAML sign-in, the trusted identity provider in the claims authentication type section must be changed.
To configure an existing web application to use the AD FS identity provider
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- In Central Administration, on the home page, click Application Management.
- On the Application Management page, in the Web Applications section, click Manage web applications.
- Click the appropriate web application.
- From the ribbon, click Authentication Providers.
- Under Zone, click the name of the zone. For example, Default.
- On the Edit Authentication page in the Claims Authentication Types section, select Trusted Identity provider, and then click the name of your SAML provider (<ProviderName> from the New-SPTrustedIdentityTokenIssuer command). Click OK.
- Next, you must enable SSL for this web application. You can do this by adding an alternate access mapping for the “https://“ version of the web application‘s URL and then configuring the web site in the Internet Information Services (IIS) Manager console for an https binding. For more information about how to set up SSL for IIS, see How to Setup SSL on IIS 7.0.
When creating a new web application to use SAML sign-in, you must configure claims authentication for the AD FS trusted identity provider. See Create claims-based web applications in SharePoint 2013 and do the following:
-
In the Security Configuration section of the New Web Application dialog box, for Use Secure Sockets Layer (SSL), select Yes.
For information about how to set up SSL for IIS, see How to Setup SSL on IIS 7.0.
- In the Claims Authentication Types section of the New Web Application dialog box, select Trusted Identity provider, and then click the name of your SAML provider (<ProviderName> from the New-SPTrustedIdentityTokenIssuer command).
-
Configure server-to-server authentication in SharePoint 2013
Updated: October 16, 2012
Summary: Find resources to help you configure server-to-server authentication for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following articles on TechNet and related resources provide information about how to configure server-to-server authentication.
-
TechNet articles about how to configure server-to-server authentication
The following articles about how to configure server-to-server authentication in SharePoint 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Configure server-to-server authentication between SharePoint 2013 farms
Describes the steps to configure server-to-server authentication between two SharePoint 2013 farms.
Configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013
Describes the steps to configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013.
Configure server-to-server authentication between SharePoint 2013 and Lync Server 2013
Describes the steps to configure server-to-server authentication between SharePoint 2013 and Lync Server 2013.
-
Configure server-to-server authentication between SharePoint 2013 farms
Published: September 4, 2012
Summary: Learn how to configure server-to-server authentication between SharePoint 2013 farms.
Applies to: SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise | SharePoint Foundation 2013
The configuration details in this article describe how to configure server-to-server authentication between SharePoint 2013 farms. For background information about server-to-server authentication, see Plan for server-to-server authentication in SharePoint 2013 Preview.
Important:
Web applications that include server-to-server authentication endpoints for incoming server-to-server requests, or that make outgoing server-to-server requests must be configured to use Secure Sockets Layer (SSL). For information about how to create a web application to use SSL, see Create claims-based web applications in SharePoint 2013.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Configure a SharePoint 2013 trust relationship with another farm
To service incoming server-to-server requests from another SharePoint 2013 farm, you must configure the SharePoint 2013 farm to trust the sending farm. Use the Windows PowerShell New-SPTrustedSecurityTokenIssuer cmdlet in SharePoint 2013 to configure the trust relationship by specifying the JavaScript Object Notation (JSON) metadata endpoint of the sending farm.
To configure a SharePoint 2013 trust relationship with another farm
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- Securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- In the SharePoint 2013 environment on the farm that is receiving server-to-server requests, start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
New-SPTrustedSecurityTokenIssuer –MetadataEndpoint “https://<HostName>/_layouts/15/metadata/json/1” –IsTrustBroker –Name “<FriendlyName>”
Where:
- <HostName> is the name and port of any SSL-enabled web application of the farm that will be sending server-to-server requests.
- <FriendlyName> is a friendly name for the sending SharePoint 2013 farm.
- Repeat step 3 for all SharePoint 2013 farms that will be sending server-to-server requests.
Note:
For more information, see New-SPTrustedSecurityTokenIssuer.
The recommended best practice for server-to-server authentication is that each server-to-server application that establishes trust with a SharePoint farm must use a different certificate. In a cross-farm SharePoint topology, if you are required to use the same certificate across the farms, you must also set the name identifier of the SharePoint Security Token Service (STS) to be the same across those farms. The following procedure describes how to synchronize the STS name identifier across two SharePoint farms.
To synchronize the STS name identifier across SharePoint farms
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- Securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- In the SharePoint 2013 environment on one of the farms, start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
Get-SPSecurityTokenServiceConfig
- In the display of the Get-SPSecurityTokenServiceConfig command, note the value of the NameIdentifier field, which starts with “00000003-0000-0ff1-ce00-000000000000@“. This is the name identifier of the SharePoint STS.
-
To set the name identifier of the SharePoint STS in the other SharePoint farm, use the following Windows PowerShell commands on a server in that farm:
$config = Get-SPSecurityTokenServiceConfig
$config.NameIdentifier=<CommonNameIdentifier>
$config.Update();
Where <CommonNameIdentifier> is the value of the NameIdentifier field from step 4.
-
Configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013
Updated: October 16, 2012
Summary: Learn how to configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Server 2013 Standard | SharePoint Foundation 2013
Server-to-server authentication enables you to share resources that live on various servers in a SharePoint farm and access services, such as Exchange Server 2013 and Lync Server 2013, which are distributed among servers. Server-to-server authentication in SharePoint 2013 also supports resource sharing and access with additional services that are compliant with the server-to-server authentication protocol.
The configuration details in this article are about how to configure server-to-server authentication between SharePoint 2013 and Exchange Server 2013.
Important:
Web applications that include server-to-server authentication endpoints for incoming server-to-server requests, or that make outgoing server-to-server requests must be configured to use Secure Sockets Layer (SSL). For information about how to create a web application to use SSL, see Create claims-based web applications in SharePoint 2013.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Process overview
This configuration has the following steps:
- Configure the SharePoint 2013 server to trust the Exchange Server 2013 server
- Configure permissions on the SharePoint 2013 server
- Configure the Exchange Server 2013 server to trust the SharePoint 2013 server
Important:
Complete the procedures in the order in which they are presented in this article.
To configure the SharePoint 2013 server to trust the Exchange Server 2013 server
- Configure the SharePoint 2013 server to trust the Exchange Server 2013 server
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following commands:
New-SPTrustedSecurityTokenIssuer –MetadataEndpoint “https://<HostName>/metadata/json/1” –IsTrustBroker –Name “<FriendlyName>”
Where:
- <HostName> is the name or address of the Exchange Server 2013 server.
- <FriendlyName> is a friendly name for the Exchange Server 2013 server.
To configure permissions on the SharePoint 2013 server
-
At the Windows PowerShell command prompt, type the following commands:
$exchange=Get-SPTrustedSecurityTokenIssuer
$app=Get-SPAppPrincipal -Site http://<HostName> -NameIdentifier $exchange.NameId
$site=Get-SPSite http://<HostName>
Set-SPAppPrincipalPermission –AppPrincipal $app –Site $site.RootWeb –Scope sitesubscription –Right fullcontrol -EnableApplyOnlyPolicy
Where:
-
<HostName> is the name or address of the SharePoint 2013 server.
Note:
For more information, see Get-SPTrustedSecurityTokenIssuer, Get-SPAppPrincipal, and Set-SPAppPrincipalPermission.
-
To configure the Exchange Server 2013 server to trust the SharePoint 2013 server
- Start the Exchange Management Shell.
-
For Windows Server 2008 R2:
- In the Exchange Server 2013 environment, on the Start menu, click All Programs, click Microsoft Exchange Server 2013, and then click Exchange Management Shell.
- In the Exchange Server 2013 environment, on the Start menu, click All Programs, click Microsoft Exchange Server 2013, and then click Exchange Management Shell.
-
For Windows Server 2012:
-
In the Exchange Server 2013 environment, on the Start screen, click Exchange Management Shell.
If Exchange Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click Exchange Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following commands:
cd c:\’Program Files’\Microsoft\’Exchange Server’\V15\Scripts
.\Configure-EnterprisePartnerApplication.ps1 -AuthMetadataUrl https://<HostName>/_layouts/15/metadata/json/1 -ApplicationType SharePoint
Where:
- <HostName> is the name and port of any SSL-enabled web application of the SharePoint farm.
Configure server-to-server authentication in SharePoint 2013
-
Configure server-to-server authentication between SharePoint 2013 and Lync Server 2013
Published: October 2, 2012
Summary: Learn how to configure server-to-server authentication between SharePoint 2013 and Lync Server 2013.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Server 2013 Standard | SharePoint Foundation 2013
Server-to-server authentication enables you to share resources that live on various servers in a SharePoint farm and access services, such as Lync Server 2013 and Exchange Server 2013, which are distributed among servers. Server-to-server authentication in SharePoint 2013 also supports resource sharing and access to additional services that are compliant with the server-to-server authentication protocol. For more information about the SharePoint server-to-server authentication protocol, see OAuth 2.0 Authentication Protocol: SharePoint Profile (http://msdn.microsoft.com/en-us/library/hh631177(office.12).aspx).
The configuration details in this article explain how to configure server-to-server authentication between SharePoint 2013 and Lync Server 2013.
Important:
Web applications that include server-to-server authentication endpoints for incoming server-to-server requests, or that make outgoing server-to-server requests must be configured to use Secure Sockets Layer (SSL). For information about how to create a web application to use SSL, see Create claims-based web applications in SharePoint 2013.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Process overview
This configuration has the following steps:
- Configure the server that runs SharePoint 2013 to trust the server that runs Lync Server 2013
- Configure the server that runs Lync Server 2013 to trust the server that runs SharePoint 2013
To configure the SharePoint 2013 server to trust the Lync Server 2013 server
- Configure the server that runs SharePoint 2013 to trust the server that runs Lync Server 2013
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen, right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following commands:
New-SPTrustedSecurityTokenIssuer –MetadataEndpoint “https://<HostName>/metadata/json/1″ –IsTrustBroker –Name “<FriendlyName>“
Where:
- <HostName> is name or address of the server that runs Lync Server 2013.
- <FriendlyName> is a friendly name for the server that runs Lync Server 2013.
To configure the Lync Server 2013 server to trust the SharePoint 2013 server
- If you have not already done this, assign a server-to-server authentication certificate to Lync Server 2013. Follow the instructions in Assigning a Server-to-Server Authentication Certificate to Microsoft Lync Server 2013.
- Configure the server that runs Lync Server 2013 for a new SharePoint partner application that corresponds to the SharePoint farm. For the instructions in Configuring an On-Premises Partner Application for Microsoft Lync Server 2013, change the metadata URL string in the embedded script from:
-
to:
-
Where:
- <NameAndPort> is the host name or address and port of any SSL-enabled web application of the SharePoint farm.
-
Configure app authentication in SharePoint Server 2013
Published: September 4, 2012
Summary: Learn how to configure app authentication in SharePoint Server 2013.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Server 2013 Standard
When you use an app for SharePoint, an external component of the app might want to access SharePoint resources. For example, a web server that is located on the intranet or the Internet might try to access a SharePoint resource. When this occurs, SharePoint has to confirm the following:
- The authentication of the identity of the app and the user on whose behalf the app is acting.
- The authorization of the access for both the app and the user whose behalf the app is acting.
App authentication is the combination of these two confirmations.
This topic describes how to configure a SharePoint Server 2013 farm for app authentication by configuring a trust, by registering the app with the Application Management service, and by configuring app permissions.
Important:
SharePoint web applications that include app authentication endpoints for incoming requests must be configured to use Secure Sockets Layer (SSL). For information about how to configure SSL for a new web application, see Create claims-based web applications in SharePoint 2013.
Note:
This topic does not apply to SharePoint Foundation 2013.
- The authentication of the identity of the app and the user on whose behalf the app is acting.
-
Process overview
This configuration has the following steps that must be performed in consecutive order:
- Configure the SharePoint Server 2013 app authentication trust.
- Register the app with the Application Management service.
- Configure app permissions.
For information about apps for SharePoint, see Overview of apps for SharePoint 2013.
Note:
Because SharePoint Server 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint Server 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
-
Step 1. Configure the SharePoint Server 2013 app authentication trust
There are two ways to configure an app authentication trust with SharePoint Server 2013:
-
If you have an Office 365 subscription and the app is also using Windows Azure Access Control Service (ACS) for authentication, you configure the SharePoint farm to trust the ACS instance that corresponds to your Office 365 subscription. ACS then acts as a common authentication broker between the on-premises SharePoint farm and the app and as the online security token service (STS). ACS generates the context tokens when the app requests access to a SharePoint resource.
In this case, configure SharePoint Server 2013 to trust ACS.
-
If you do not have an Office 365 subscription or if the app does not use ACS for authentication, you must configure a server-to-server trust relationship between the SharePoint farm and the app, known as a high-trust app. A high-trust app generates its own context tokens when it requests access to a SharePoint resource. This must be done for each high-trust app that a SharePoint farm must trust. For example, if multiple apps are running on one server and if they all use different token signing certificates, you must create a separate trust with each one.
In this case, configure SharePoint Server 2013 to trust the app.
-
Configure SharePoint Server 2013 to trust ACS
-
Use the following procedure to configure SharePoint Server 2013 to trust ACS.
To configure a SharePoint Server 2013 trust relationship with ACS
-
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint Server 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- In the SharePoint 2013 environment, on the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
In the SharePoint 2013 environment, on the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
$New-SPTrustedSecurityTokenIssuer –MetadataEndpoint “<Metadata endpoint URL of ACS>” –IsTrustBroker –Name “ACS”
Where:
- <Metadata endpoint URL of ACS> for SharePoint Server 2013 is https://accounts.accesscontrol.windows.net/metadata/json/1/?realm=<contextID property of your Office 365 subscription>.
- Keep the Windows PowerShell command prompt open for the Step 2. Register the app with the Application Management service.
-
Configure SharePoint Server 2013 to trust the app
Use the following procedure to configure SharePoint Server 2013 to trust the app.
To configure a SharePoint Server 2013 trust relationship with a high-trust app
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint Server 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- In Central Administration on the SharePoint Server 2013 server in the farm, on the Quick Launch, click System Settings, and then click Manage services on server.
- In the list of services on the server, make sure that that User Profile Service is started.
- In Central Administration, on the Quick Launch, click Application Management, and then click Manage service applications.
- In the list of service applications, make sure that that the App Management Service and User Profile Service Application are started.
- Obtain a .CER version of the signing certificate of the high-trust app and store it in a location that can be accessed during the rest of this procedure.
- Verify that you are a member of the Administrators group on the server on which you are running Windows PowerShell cmdlets.
- securityadmin fixed server role on the SQL Server instance.
-
db_owner fixed database role on all databases that are to be updated.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint Server 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Click Start menu, click All Programs, click SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands:
$appId = “<AppID>“
$spweb = Get-SPWeb “<AppURL>“
$realm = Get-SPAuthenticationRealm -ServiceContext $spweb.Site
$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(“<CERFilePath>“)
$fullAppIdentifier = $appId + ‘@’ + $realm
New-SPTrustedSecurityTokenIssuer -Name “<FriendlyName>” -Certificate $certificate -RegisteredIssuerName $fullAppIdentifier
Where:
-
<AppID> is the client ID assigned to the high-trust app when it was created.
Important:
All of the letters in the AppID must be in lowercase.
- <AppURL> is the URL to the high-trust app‘s location on the app server.
- <CERFilePath> is the path of the .CER version of the signing certificate of the high-trust app.
- <FriendlyName> is a friendly name that identifies the app.
- Keep the Windows PowerShell command prompt open for the next procedure.
-
Step 2. Register the app with the Application Management service
Use the following procedure to register the app with the Application Management service.
To register the app as a SharePoint app principal
-
At the Windows PowerShell command prompt, type the following command:
$appPrincipal = Register-SPAppPrincipal -NameIdentifier $fullAppIdentifier -Site $spweb -DisplayName “<DisplayName>”
Where:
- <DisplayName> is the name of the app as displayed in Central Administration.
- Keep the Windows PowerShell command prompt open for the next procedure.
-
Step 3. Configure app permissions
Use the following Windows PowerShell command to add or change individual app permissions. Repeat this procedure for as many times as needed to configure the permissions of the app.
To configure app permissions
-
At the Windows PowerShell command prompt, type the following command:
Set-AppPrincipalPermission -appPrincipal $appPrincipal -site $web -right <Level> -scope <Scope>
Where:
- <Level> is Read, Write, Manage, or FullControl.
-
<Scope> is Farm, Site collection, SharePoint Online, Web, Documents, List, or Library.
For more information, see Set-SPAppPrincipalPermission
- <Level> is Read, Write, Manage, or FullControl.
For more information, see Plan app permissions management in SharePoint 2013.
-
-
Configure client certificate authentication for SharePoint 2013
Published: September 25, 2012
Summary: Learn how to configure SharePoint 2013 to support user authentication using a client certificate.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Server 2013 Standard | SharePoint Foundation 2013
Client certificate authentication enables web-based clients to establish their identity to a server by using a digital certificate, which provides additional security for user authentication. SharePoint 2013 does not provide built-in support for client certificate authentication, but client certificate authentication is available through Security Assertion Markup Language (SAML)-based claims authentication. You can use Active Directory Federation Services (AD FS) 2.0 as your security token service (STS) for SAML claims or any third-party identity management system that supports standard security protocols such as WS-Trust, WS-Federation, and SAML 1.1.
Note:
For more information about SharePoint 2013 protocol requirements, see SharePoint Front-End Protocols.
Claims-based authentication in SharePoint 2013 allows you to use different STSs. If you configure AD FS as your STS, SharePoint 2013 can support any identity provider or authentication method that AD FS supports, which includes client certificate authentication.
Note:
For more information about AD FS, see Active Directory Federation Services Overview.
In the following figure, SharePoint 2013 is configured as a relying partner for an AD FS-based STS.
AD FS can authenticate user accounts for several different types of authentication methods, such as forms-based authentication, Active Directory Domain Services (AD DS), client certificates, and smart cards. When you configure SharePoint 2013 as a relying partner of AD FS, SharePoint 2013 trusts the accounts that AD FS validates and the authentication methods that AD FS uses to validate those accounts. This is how SharePoint 2013 supports client certificate authentication.
-
Configure client certificate authentication
The following topics explain how to configure SharePoint 2013 with client certificate authentication or smart card authentication when you use AD FS as your STS:
-
Configure AD FS to support claims-based authentication.
For more information, see AD FS 2.0 – How to change the local authentication type (http://go.microsoft.com/fwlink/p/?LinkId=212513).
-
Configure SharePoint 2013 to support SAML-based claims authentication using AD FS.
For more information, see Configure SAML-based claims authentication with AD FS in SharePoint 2013.
-
Create a web application that uses SAML-based claims authentication.
For more information, see Create claims-based web applications in SharePoint 2013.
Note:
These steps will be similar for a third-party STS.
-
Configure availability and recovery solutions for SharePoint 2013
Published: October 16, 2012
Summary: Learn about the options for providing high availability and disaster recovery solutions for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about installing and configuring high availability and disaster recovery solutions for a SharePoint 2013 farm.
The articles in this section assume that you are familiar with the concepts and terms presented in High availability and disaster recovery concepts in SharePoint 2013 and Plan for high availability and disaster recovery for SharePoint 2013.
-
TechNet articles about installing and configuring high availability and disaster recovery solutions
The following articles about high availability and disaster recovery solutions are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Configure SQL Server 2012 AlwaysOn Availability Groups for SharePoint 2013
Describes how to install and configure a SQL Server 2012 AlwaysOn Availability Group for a SharePoint farm.
-
Configure SQL Server 2012 AlwaysOn Availability Groups for SharePoint 2013
Published: October 16, 2012
Summary: Learn how to create and configure a SQL Server 2012 AlwaysOn Availability Group for a SharePoint 2013 farm.
Applies to: SharePoint Server 2013 | SharePoint Server 2013 Enterprise
This article provides the required information and detailed procedures to create and configure a SQL Server 2012 AlwaysOn Availability Group for a SharePoint 2013 farm.
Important:
The steps in this article apply to both SharePoint Foundation 2013 and SharePoint Server 2013. With both of these products these steps are to deploy a new SharePoint farm and do not cover upgrading from SQL Server 2008 R2 to SQL Server 2012.
In this article:
-
Process overview
We recommend the following installation sequence and key configuration steps to deploy a SharePoint farm that uses an AlwaysOn availability group:
- Select or create a Windows Server failover cluster.
- Install SQL Server 2012 on each cluster node.
- Create and configure an availability group.
- Install and configure SharePoint 2013.
- Add the SharePoint databases to the availability group.
- Test failover for the availability group.
- Select or create a Windows Server failover cluster.
-
Before you begin
A SQL Server 2012 AlwaysOn Availability Group is not just a combination of database mirroring and database clustering. It is a completely new high availability and disaster recovery feature that co-exists with existing high availability and disaster recover options such as mirroring and log shipping.
Before you begin deployment, review the following information about SQL Server AlwaysOn, the technologies that support AlwaysOn, and SharePoint 2013:
- Knowledge and skill requirements
- AlwaysOn Availability Group concepts
- Hardware and software requirements
- Permissions
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
-
Knowledge and skill requirements
To implement SQL Server AlwaysOn Availability Groups as a high availability and disaster recovery solution, several technologies interact and have to be installed and configured correctly. We recommend that the team responsible for setting up an AlwaysOn environment for SharePoint products has a working knowledge of, and hands-on skills with the following technologies:
- Windows Server Failover Clustering (WSFC) services
- SQL Server 2012
-
SharePoint 2013
-
SQL Server AlwaysOn Availability Group concepts
-
A SQL Server Availability Group enables you to specify a set of databases that you want to fail over together as a single entity. When an availability group fails over to a target instance or target server, all the databases in the group fail over also. Because SQL Server 2012 can host multiple availability groups on a single server, you can configure AlwaysOn to fail over to SQL Server instances on different servers. This reduces the need to have idle high performance standby servers to handle the full load of the primary server, which is one of the many benefits of using availability groups.
An availability group consists of the following components:
- Replicas, which are a discrete set of user databases called availability databases that fail over together as a single unit. Every availability group supports one primary replica and up to four secondary replicas.
- A specific instance of SQL Server to host each replica and to maintain a local copy of each database that belongs to the availability group.
Replicas and failover
The primary replica makes the availability databases available for read-write connections from clients and sends transaction log records for each primary database to every secondary replica. Each secondary replica applies transaction log records to its secondary databases.
All replicas can run under asynchronous-commit mode, or up to three of them can run under synchronous-commit mode. For more information about synchronous and asynchronous commit mode, see Availability Modes (AlwaysOn Availability Groups).
Note:
Database issues, such as a database becoming suspect due to a loss of a data file, deletion of a database, or corruption of a transaction log do not cause failovers.
Read the following articles to learn required and important concepts about SQL Server AlwaysOn technology:
- For details about the benefits of AlwaysOn Availability Groups and an overview of AlwaysOn Availability Groups terminology, see AlwaysOn Availability Groups (SQL Server).
-
For detailed information about prerequisites, see Prerequisites, Restrictions, and Recommendations for AlwaysOn Availability Groups (SQL Server). This article contains the following information:
- Windows Server system requirements and recommendations
-
SQL Server instance prerequisites and restrictions
Important:
You can install SQL Server 2012 on Windows Server core to improve security and reduce maintenance, but you cannot install SharePoint 2013 on Windows Server core. For more information, see Server Core for Windows Server 2008 R2 [Server Core for Windows Server 2008 R2. For information about server core and Windows Server 2012, see Windows Server Installation Options.
- Prerequisites and restrictions for using a SQL Server Failover Cluster Instance (FCI) to host an availability replica
- Windows Server system requirements and recommendations
- Availability group prerequisites and restrictions
-
Availability database prerequisites and restrictions
-
Windows Server Failover Clustering
-
To create and use SQL Server 2012 AlwaysOn Availability Groups, you have to install SQL Server 2012 on a Windows Server Failover Clustering (WSFC) cluster. For more information, see Windows Server Failover Clustering (WSFC) with SQL Server.
Although configuring a WSFC cluster is out of the scope for this article, you should be aware of the following requirements before you install and configure a cluster:
- All the cluster nodes must be in the same Active Directory Domain Services (AD DS) domain.
- Each availability replica in an availability group must reside on a different node of the same Windows Server Failover Clustering (WSFC) cluster.
-
The cluster creator must have the following accounts and permissions:
- Have a domain account in the domain where the cluster will exist.
- Have local administrator permissions on each cluster node.
- Have Create Computer objects and Read All Properties permissions in AD DS. For more information, see Failover Cluster Step-by-Step Guide: Configuring Accounts in Active Directory or Active Directory Domain Services for Windows Server 2012.
- Have a domain account in the domain where the cluster will exist.
A very important aspect of configuring failover clustering and AlwaysOn is determining the quorum votes that are needed for the cluster nodes.
Failover clustering is based on a voting algorithm where more than one half of the voters, or quorum, must be online and able to communicate with each other. Because a given cluster has a specific number of nodes and a specific quorum configuration, the cluster service is able to determine what constitutes a quorum. The cluster service will stop on all the nodes if the number of voters drops below the required majority.
For more information, see WSFC Quorum Modes and Voting Configuration (SQL Server) and Configure Cluster Quorum NodeWeight Settings.
-
SharePoint Foundation 2013 and SharePoint Server 2013
Some SharePoint 2013 databases do not support SQL Server AlwaysOn Availability Groups. We recommend that you review the Supported high availability and disaster recovery options for SharePoint databases before you configure an AlwaysOn environment. You should also review the Hardware and software requirements for SharePoint 2013 article.
- Knowledge and skill requirements
-
Detailed steps to configure an AlwaysOn Availability Group for SharePoint
The following illustration shows a SharePoint 2013 farm (SPHA_farm) that uses an availability group named SP_AG1. We’ll use SPHA_farm as reference example in our steps to configure AlwaysOn.
-
Prepare the Windows Server cluster environment
Obtain access to or create a three node Windows Server Failover Clustering (WSFC) cluster that you can use to install SQL Server 2012 on each cluster node. The following reference material provides guidance and detailed steps to configure a Windows Server failover cluster:
-
This page provides links to Getting Started, Deployment, Operations, and Troubleshooting articles for Windows Server 2008 R2.
-
This page provides links to Getting Started, Deployment, Operations, and Troubleshooting articles for Windows Server 2012.
-
Prepare the SQL Server environment
-
Before you can create an Availability Group for SharePoint Foundation 2013 or SharePoint Server 2013, you must prepare the SQL Server 2012 environment. To prepare the environment, complete the following tasks:
- Install the SQL Server prerequisites.
- Install SQL Server.
- Enable Named Pipes.
- Enable AlwaysOn.
When you prepare the database server environment you must consider SharePoint 2013 database requirements. Refer to the following articles before you install SQL Server:
- Hardware requirements—database servers
- Install SharePoint 2013
- Configure SQL Server security for SharePoint 2013 environments
-
Supportability regarding SQL collation for SharePoint Databases and TempDB. (This guidance in this article also applies to SharePoint 2013.)
-
Install SQL Server 2012
-
To install SQL Server 2012
-
-
Install SQL Server 2012 prerequisites on each cluster node.
For more information, see Prerequisites, Restrictions, and Recommendations for AlwaysOn Availability Groups (SQL Server).
-
Install SQL Server on each cluster node.
For more information, see Installation for SQL Server 2012.
-
Enable Named Pipes
Named Pipes is required for an AlwaysOn Availability Group. Use the following procedure to enable Named Pipes for SQL Server.
To enable Named Pipes
- Make sure that the logon has the required credentials. To change a network configuration for the database engine, you must be a member of the sysadmin fixed server role.
- Log on to the server that will host the primary replica and start SQL Server Management Studio.
- Expand SQL Server Network Configuration and then click Protocols for<instance name>.
- In the details pane, right-click Named Pipes and then click Enable from the list of available options.
- In the console pane, click SQL Server Services.
- In the details pane, right-click SQL Server ( <instance name> ) and then click Restart, to stop and restart SQL Server.
- Repeat the previous steps to enable Named Pipes for SQL Server on the other cluster nodes.
-
Enable AlwaysOn
After you enable Named Pipes, you must enable AlwaysOn for each of the database servers in the cluster.
Note:
You can enable AlwaysOn by using SQL Server Management Studio, Transact-SQL, or Windows PowerShell 3.0.
To enable AlwaysOn
- Make sure that your logon account has the required permissions to create an availability group. The account must have membership in the db_owner fixed database role and either CREATE AVAILABILITY GROUP server permission, CONTROL AVAILABILITY GROUP permission, ALTER ANY AVAILABILITY GROUP permission, or CONTROL SERVER permission.
- Log on to the server that will host the primary replica and start SQL Server Management Studio.
- In Object Explorer, select SQL Server Services, right-click SQL Server (<instance name>), where <instance name> is the name of a local server instance for which you want to enable AlwaysOn Availability Groups, and then click Properties.
- Select the AlwaysOn High Availability tab.
- Select the Enable AlwaysOn Availability Groups check box, and then click OK.
- Although the change is saved you must manually restart the SQL Server service (MSSQLSERVER) to commit the change. The manual restart enables you to choose a restart time that is best for your business requirements.
- Repeat the previous steps to enable AlwaysOn for SQL Server on the other cluster nodes.
For more information, see Enable and Disable AlwaysOn Availability Groups (SQL Server).
-
Create and configure the availability group
Depending on the SQL Server 2012 environment where you plan to create the Availability Group, you might have to create a temporary database to before you create the Availability Group.
The process that creates an availability group requires you to provide a name for the availability group and then select an eligible user database on the connected server instance as an availability database.
Note:
To be eligible to be added to an availability group, a database must be a user database. System databases cannot belong to an availability group. For more information, see the “Availability Database Prerequisites and Restrictions” section of Prerequisites, Restrictions, and Recommendations for AlwaysOn Availability Groups (SQL Server) and see Creation and Configuration of Availability Groups (SQL Server).
If there no user databases are on the instance of the connected server, which is the case in our example, you need to create one.
Use the following procedure to create a temporary user database that will be a temporary primary replica for the group.
To create a temporary user database
- Make sure that your logon account has the correct permissions for this task. You require one of the following permissions in the master database to create the new database:
- CREATE DATABASE
- CREATE ANY DATABASE
- ALTER ANY DATABASE
- Log on to the server that will host the primary replica, which is SP-SRV1 in our example.
- Start Management Studio.
- In Object Explorer, right-click Databases and then click New Database.
-
In the New Database dialog box, type the Database name:, which is “TemporaryUserDB” for this example.
Because this is a temporary database that you delete after you create the availability group, you can use the default settings. Click OK.
Because the New Availability Group Wizard will not create an availability group unless the user database was backed up, you have to back up the temporary database.
- In Object Explorer expand Databases and right-click the temporary database that you just created. Pick Tasks and then choose Back Up.
- In the Back Up Database dialog box, click OK to accept all the default settings and create the back up.
-
About replicas and data synchronization
About replicas
Every availability replica is assigned an initial role—either the primary role or the secondary role, which the availability databases of that replica inherit. The role of a given replica determines whether it hosts read-write databases or read-only databases, the type of failover and whether it uses synchronous commit or asynchronous commit.
The following table shows the information that you have to provide for each replica, either when you first create the availability group, or when you add secondary replicas.
-
Replica configuration requirements
|
Replica information |
Description |
|
Server Instance |
Displays the name of the instance of the server that will host the availability replica. |
|
Initial Role |
Indicates the role that the new replica will first perform: Primary or Secondary. |
|
Automatic Failover (Up to 2) |
Indicates the type of failover that the replica uses: automatic or manual. |
|
Synchronous Commit (Up to 3) |
Indicates the type of commit that is used for the replica. |
|
Readable Secondary |
Indicates whether a secondary replica can be read. The configuration options are unavailable for read access, read-only, and read-only intent. For more information, see Readable Secondary Replicas (AlwaysOn Availability Groups).
Readable secondary replicas are currently not supported for SharePoint 2013 runtime usage. |
Note:
When you add replicas to a group, you will also provide the endpoint for each replica and configure backup preferences. For more information, see Specify the Endpoint URL When Adding or Modifying an Availability Replica (SQL Server) and Backup on Secondary Replicas (AlwaysOn Availability Groups).
Data synchronization
As part of the availability group creation process, you have to make an exact copy of the data on the primary replica and install the copy on the secondary replica. This is the initial data synchronization for the Availability Group. For more information, see Select Initial Data Synchronization Page (AlwaysOn Availability Group Wizards).
A network share must exist and must be accessed by all the nodes in the AlwaysOn configuration to do the initial data synchronization between all the cluster nodes that host a replica. For more information, see Network Shares Extension and File Services.
The following restrictions exist when you use the New Availability Group wizard to start data synchronization:
- If the file paths on the secondary replica location differ from the file paths on the primary location, you have to start data synchronization manually.
- If any secondary database exists on a secondary replica, you have to manually delete the secondary databases before you start data synchronization in the New Availability Group. However, if you want to use existing secondary databases, exit the New Availability Group wizard and start data synchronization manually.
- To use the availability group wizard to synchronize data, you have to have a backup share that all the replicas can write to. You can specify the share by browsing to it or by entering its fully qualified universal naming convention (UNC) path name, \\Systemname\ShareName\Path\, in the Specify a shared network location accessible by all replicas box.
For each database in the availability group, the Start Data Synchronization page shows the progress of the following operations:
- Creating a full database backup of the primary database on the network share.
- Creating a full database backup of the primary database on the network share.
-
Restoring these backups to the secondary replica location.
These restore operations both use RESTORE WITH NORECOVERY option and leave the new secondary database in the RESTORING state.
-
Joining the secondary database to the availability group.
This step puts the secondary database in the ONLINE state and starts data synchronization for this database.
Login replication
SharePoint logins that are created by using the same approach as in previous releases of SQL Server are not replicated in an availability group. This occurs because login information is stored in the MasterDB database, which is not replicated. Although the farm accounts are created when replicas are synchronized, login information is not available after a failover.
If you have already created an availability group and synchronized the primary and secondary replicas, the workaround is to manually copy the logins from the primary replica to the secondary replicas.
SQL Server 2012 introduces the concept of Users with Passwords for Contained Databases. The database itself stores all the database metadata and user information, and a user who is defined in this database does not have to have a corresponding login. The information in this database is replicated by the availability group and is available after a failover. For more information, see Contained Databases.
Important:
If you create a new SharePoint login to use for an existing availability group, make sure to add the login to the contained database so it is replicated to each server that is hosting a SQL Server instance for the availability group. For example, if you create another application pool for a Web App and give it a new identity (an application pool account that you have not used), then you need to add that account as a login.
-
Create and configure the availability group
Use the following procedure to create an availability group on the primary replica, which is SP-SRV1 in our example.
-
Create the availability group
- Make sure that your logon account has the required permissions to create an availability group. This requires membership in the db_owner fixed database role and either CREATE AVAILABILITY GROUP server permission, CONTROL AVAILABILITY GROUP permission, ALTER ANY AVAILABILITY GROUP permission, or CONTROL SERVER permission.
- Log on to the server that will host the primary replica and start SQL Server Management Studio.
- To start the New Availability Group Wizard, right-click AlwaysOn High Availability and then click New Availability Group Wizard.
-
Click Next to advance to the Specify Name page. Enter SP-AG1 as the name of the new availability group in the Availability group name: box.
This name must be: a valid SQL Server identifier, unique on the Windows Server Failover Clustering cluster and unique on the domain.
- On the Select Databases page, all user databases that are eligible to become the primary database for the new availability group are listed on the User databases on this instance of SQL Server grid. Select TemporaryUserDB, and then click Next.
- On the Specify Replicas page, use the following tabs to configure the replicas for SP-AG1: Replicas, Endpoints, and Backup Preferences.
-
On the Listener tab, configure an availability group listener for our example.
An availability group listener is a server name to which clients can connect r to access a database in a primary or secondary replica of an availability group. Availability group listeners direct incoming connections to the primary replica or to a read-only secondary replica. The listener provides fast application failover after an availability group fails over. For more information, see Availability Group Listeners, Client Connectivity, and Application Failover (SQL Server).
Important:
Intermittent, unusually high latency might occur when you use availability groups that have replicas that are deployed on multiple subnets.
As a best practice, connections to SharePoint availability groups in a multi-subnet environment should configure specifyMultiSubnetFailover=True to avoid issues caused by high network latency. For more information, see Supporting Availability Group Multi-Subnet Failovers.
You cannot directly specify MultiSubnetFailover=True because a SharePoint client cannot directly modify a connection string. You must use Windows PowerShell to set this value on the MultiSubnetFailover database property. The following example shows how to do this.
C#
$dbs = Get-SPDatabase | ?{$_.MultiSubnetFailover –ne $true}
foreach ($db in $dbs)
{
$db.MultiSubnetFailover = $true
$db.Update()
}
- Select the desired configuration for each instance in the Selected instances grid, and then click Next.
- Click Finish to create the availability group.
- The Select Initial Data Synchronization page lets you select a synchronization preference and specify the shared network location that all replicas can access. For our environment accept the default, Full, which performs full database and log backups. Click Next.
- The Validation page of the wizard displays the results of six checks before it lets you continue with availability group creation. If all checks pass, click Next to continue. If any tests fail, you cannot continue until you correct the error and then click Re-run Validation to run the validation tests again. When all the tests pass, click Next to continue.
- On the Summary page, verify the configuration of the replica that you are adding and then click Finish to save it. To change the configuration, click Previous to return to previous wizard pages.
-
Install and configure SharePoint 2013
At this point in the process, you can install SharePoint 2013 and create the farm. Use the following procedure as a guide to install and configure SharePoint 2013.
Note:
For detailed installation and configuration instructions, see Prepare for installation of SharePoint 2013 and Install SharePoint 2013.
To install SharePoint 2013
- Copy the SharePoint 2013 program files to a local disk on the computer where you plan to install SharePoint products or to a network file share.
- Run the Microsoft SharePoint Products Preparation Tool to install all the prerequisites to set up and use SharePoint 2013.
- Run Setup to install binaries, configure security permissions, and edit registry settings for SharePoint 2013.
- Run the SharePoint Products Configuration Wizard to install and configure the configuration database, install and configure the content database, and install the SharePoint Central Administration website.
Note:
When you run the configuration wizard, you have to identify the server that will host the SharePoint databases. On the Specify Configuration Database Settings page, in the Database server box, type SP-SRV1 as the name of the computer that is running SQL Server.
To finalize setup of AlwaysOn for a SharePoint 2013 farm, add the SharePoint databases to the availability group and synchronize secondary replicas to the primary replica.
Important:
Only add the databases that are supported for use with a SQL Server AlwaysOn Availability Group.
On the server that hosts the primary replica, you have to run the Add Databases to Availability Group wizard to add all the SharePoint databases to the availability group. The following procedure is the same as the procedure that we described to create the availability group.
To add SharePoint databases to the availability group
-
Log on to the server that will host the primary replica and start SQL Server Management Studio.
The account that that you use must be a member of the Local Administrators group for each server where you install SharePoint 2013
In addition, the account must have at least one of the following permissions:
- ALTER AVAILABILITY GROUP permission on the availability group
- CONTROL AVAILABILITY GROUP permission
- ALTER ANY AVAILABILITY GROUP permission
-
CONTROL SERVER permission
To join a database to availability group requires membership in the db_owner fixed database role.
- In Object Explorer, browse to, and if it is necessary expand the Availability Groups.
- Right-click the example group, SP-AG1, and then click Add Database.
- On the Select Databases page, all user databases that are eligible to become the primary database for the new availability group are listed on the User databases on this instance of SQL Server grid. Use the checkboxes to select all the databases that you want to add to the group, and then click Next.
- The Select Initial Data Synchronization page lets you select a synchronization preference and specify the shared network location that all replicas can access. For our environment we’ll accept the default, Full, which performs full database and log backups. Click Next.
- The Validation page of the wizard displays the results of six checks before it lets you continue with availability group creation. If any tests fail, you cannot continue until you correct the error and then click Re-run Validation to run the validation tests again. When all the tests pass, click Next to continue.
- On the Summary page, verify the configuration of the replica that you are adding, and then click Finish to keep it. To change the configuration, click Previous to return to previous wizard pages.
Important:
Databases that you add to a SharePoint farm are not automatically added to the availability group. You must add them by using the steps described in this article or by using scripts to automate the procedure.
-
Use failover tests to validate the AlwaysOn installation
After you synchronize the SharePoint data with the secondary replicas, the final step is to test failover.
You must run extensive failover tests to make sure that the behavior of the AlwaysOn environment is as expected and that you completely understand the configuration requirements and procedures related to SQL Server 2012Availability Groups. These tests include and are not limited to the following:
- Verify that all the farm services and features are completely functional.
- Verify that SharePoint 2013 data is preserved and not corrupted.
Test availability group failover by using either the planned manual failover described in Perform a Planned Manual Failover of an Availability Group (SQL Server) or the forced manual failover described in Perform a Forced Manual Failover of an Availability Group (SQL Server).
You can perform either of the previous failovers by using the Failover Wizard in SQL Server Management Studio, Transact-SQL, or Windows PowerShell in SQL Server 2012.
Note:
In an Active-Active failover cluster scenario where there are multiple SharePoint instances can fail over to each other you must ensure that each server has enough capacity to handle the local workload and the workload from the failed server.
- Verify that all the farm services and features are completely functional.
-
Monitor the AlwaysOn environment
You have to monitor an AlwaysOn environment for performance, health, and capacity.
Performance
New performance objects, SQLServer:Database Replica and SQLServer:Availability Replica, are available to monitor an AlwaysOn environment.
Health and capacity
For general health monitoring you can use the Availability Groups Dashboard to obtain the health of the availability groups in the system. We recommend that you refer to the following posts on the official SQL Server AlwaysOn team blog to fully understand AlwaysOn health monitoring.
- The AlwaysOn Health Model Part 1 — Health Model Architecture
- The AlwaysOn Health Model Part 2 — Extending the Health Model
You can also use Transact-SQL to monitor availability groups by using the set of catalog and dynamic management views that are provided for AlwaysOn Availability Groups. For more information, see Monitor Availability Groups (Transact-SQL).
- The AlwaysOn Health Model Part 1 — Health Model Architecture
-
Configure email integration for a SharePoint 2013 farm
Published: July 16, 2012
Summary: Use these TechNet articles to learn how to configure incoming and outgoing email for a SharePoint 2013 farm.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following articles on TechNet provide information about email integration. After you install SharePoint 2013, you can configure incoming and outgoing email. These optional settings are useful if you want to work with email in the server farm.
-
TechNet articles about email integration
The following articles about email integration are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
This article describes how to configure incoming email so that SharePoint 2013 sites accept and archive incoming email. It also describes how to configure incoming email so that SharePoint sites can archive email discussions as they happen, save attachments, and show meetings that were sent and received by email on site calendars. In addition, this article describes how to configure the SharePoint Directory Management Service to provide support for email distribution list creation and management.
This article describes how to configure outgoing email so that your Simple Mail Transfer Protocol (SMTP) server sends email alerts to site users and notifications to site administrators.
-
Configure incoming email for a SharePoint 2013 farm
Published: July 16, 2012
Summary: Learn how to install and configure the SMTP service, prepare your environment, and configure incoming email for a SharePoint 2013 farm.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article describes how to configure incoming email for SharePoint 2013. This article also describes how to install and configure the SMTP service that you must use to enable incoming email.
When incoming email is enabled, SharePoint sites can receive and store email messages and attachments in lists and libraries. This article describes two scenarios, one basic and one advanced. The basic scenario applies to a single-server farm environment and is recommended if you want to use default settings. The advanced scenario applies to a single-server farm or a multiple-server farm and contains several advanced options from which to choose. For more information, see Plan incoming email (SharePoint 2013 Preview).
In this article:
- Before you begin
- Install and configure the SMTP service
- Configure incoming email in a basic scenario
- Configure incoming email in an advanced scenario
- Prepare your environment for incoming email in an advanced scenario
- Are attachments missing from email messages that are sent to a SharePoint document library?
- Before you begin
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
Before you begin this operation, review the following information about prerequisites:
- Your system is running SharePoint 2013.
- You have read Plan incoming email (SharePoint 2013 Preview).
- For the basic scenario, each SharePoint front-end web server must be running the Simple Mail Transfer Protocol (SMTP) service and the SharePoint Foundation Web Application service.
- For the advanced scenario, you can use one or more servers in the server farm to run the SMTP service and to have a valid SMTP server address. Alternatively, you must know the name of a server outside the farm that is running the SMTP service and the location of the email drop folder.
If you have not installed and configured the SMTP service and do not choose to use an email drop folder, you must complete the steps in Install and configure the SMTP service before you configure incoming email.
- Plan browser support
-
Install and configure the SMTP service
Incoming email for SharePoint 2013 uses the SMTP service. You can use the SMTP service in one of two ways. You can install the SMTP service on one or more servers in the farm, or administrators can provide an email drop folder for email that is forwarded from the service on another server. For more information about the email drop folder option, see Plan incoming email (SharePoint 2013 Preview).
-
Install the SMTP service
If you are not using a drop folder for email, the SMTP service must be installed on every front-end web server in the farm that you want to configure for incoming email. To install the SMTP service, use the Add Features Wizard in Server Manager. After you complete the procedure, the SMTP service is installed on the front-end web server.
To install the SMTP service
-
- Verify that the user account that is performing this procedure is a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Server Manager.
- In Server Manager, click Features.
- In Features Summary, click Add Features to open the Add Features Wizard.
- On the Select Features page, select SMTP Server.
- In the Add Features Wizard dialog box, click Add Required Roll Services, and then click Next.
- On the Confirm Installation Selections page, click Install.
- On the Installation Results page, ensure that the installation finished successfully, and then click Close.
-
Install IIS 6.0 Management tools
To manage the SMTP service on Windows Server 2008 and Windows Server 2008 R2, you must use Internet Information Services (IIS) 6.0 Manager.
To install IIS 6.0 Manager
- Verify that you have the following administrative credentials:
- You must be a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Server Manager.
- In Server Manager, click Roles.
- In Application Server section, click Add Role Services.
- On the Select Role Services page, select Management Tools and IIS 6 Management compatibility, and then click Install.
-
Configure the SMTP service
After you install the SMTP service, you configure it to accept email from the mail server for the domain. You can decide to accept relayed email from all servers except those that you specifically exclude. Alternatively, you can block email from all servers except those that you specifically include. You can include servers individually, in groups by subnet, or in groups by domain.
After you configure the service, set it to start automatically.
To configure the SMTP service
- Verify that the user account that is performing this procedure is a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) 6.0 Manager.
- In IIS Manager, expand the server name that contains the SMTP server that you want to configure.
- Right-click the SMTP virtual server that you want to configure, and then click Start.
- Right-click the SMTP virtual server that you want to configure, and then click Properties.
- On the Access tab, in the Access control area, click Authentication.
- In the Authentication dialog box, verify that Anonymous access is selected.
- Click OK.
- On the Access tab, in the Relay restrictions area, click Relay.
- To enable relaying from any server, click All except the list below.
-
To accept relaying from one or more specific servers, follow these steps:
- Click Only the list below.
- Click Add, and then add servers one at a time by IP address, or in groups by using a subnet or domain.
- Click OK to close the Computer dialog box.
- Click Only the list below.
- Click OK to close the Relay Restrictions dialog box.
- Click OK to close the Properties dialog box.
To set the SMTP service to start automatically
- Click Start, point to Administrative Tools, and then click Services.
- In Services, right-click Simple Mail Transfer Protocol (SMTP), and then select Properties.
- In the Simple Mail Transfer Protocol (SMTP) Properties dialog box, on the General tab, in the Startup type list, select Automatic.
- Click OK.
-
Configure incoming email in a basic scenario
You can use the following procedure to configure incoming email in a basic scenario by selecting the Automatic settings mode and using the default settings. After you complete the procedure, users can send email to lists and libraries.
To configure incoming email in a basic scenario
- Verify that the user account that is performing this procedure is a member of the Administrators group on the server that is running the SharePoint Central Administration website.
- In Central Administration, click System Settings.
- On the System Settings page, in the E-Mail and Text Messages (SMS) section, click Configure incoming e-mail settings.
- If you want to enable sites on this server to receive email, on the Configure Incoming E-Mail Settings page, in the Enable Incoming E-Mail section, click Yes.
- Select the Automatic settings mode.
- In the Incoming E-Mail Server Display Address section, in the E-mail server display address box, type a display name for the email server, for example, mail.fabrikam.com.
- Use the default settings for all other sections, and then click OK.
After you configure incoming email, users who have Manage Lists permissions can configure email–enabled lists and document libraries.
-
Configure incoming email in an advanced scenario
You can use the following procedure to configure incoming email in an advanced scenario by selecting the Advanced settings mode and additional options that you want to use for your incoming email environment. After you complete the procedure, users can send email to lists and libraries.
You can also use the Automatic settings mode in an advanced scenario. In the Automatic settings mode, you can select to receive email that has been routed through a safe-email server application. In the Advanced settings mode, you can instead specify a drop folder. For more information, see Plan incoming email (SharePoint 2013 Preview).
Several of these steps mention prerequisite procedures that are documented in Prepare your environment for incoming email in an advanced scenario later in this article.
- Verify that the user account that is performing this procedure is a member of the Administrators group on the server that is running the SharePoint Central Administration website.
- In Central Administration, click System Settings.
- On the System Settings page, in the E-Mail and Text Messages (SMS) section, click Configure incoming e-mail settings.
- If you want to enable sites on this server to receive email, on the Configure Incoming E-mail Settings page, in the Enable Incoming E-Mail section, click Yes.
-
Select the Advanced settings mode.
You can specify a drop folder instead of using an SMTP server.
Note:
You can also select the Automatic settings mode and select whether to use Directory Management Service and whether to accept email from all email servers or from several specified email servers. For more information, see Plan incoming email (SharePoint 2013 Preview).
-
If you want to connect to Directory Management Service, in the Directory Management Service section, click Yes.
If you select this option, you must first configure Active Directory Domain Services (AD DS). If you use Exchange Server, you must also configure the DNS Manager and add an SMTP connector. For more information, see Configure AD DS to be used with Directory Management Service, Configure DNS Manager, and Add an SMTP connector in Microsoft Exchange Server 2010 later in this article.
- In the Active Directory container where new distribution groups and contacts will be created box, type the name of the container in the format OU=ContainerName, DC=domain, DC=com, where ContainerName is the name of the OU in AD DS, domain is the second-level domain, and com is the top-level domain.
The application pool identity account for Central Administration must be delegated the Create, delete, and manage user accounts task for the container. Access is configured in the properties for the OU in AD DS.
- In the SMTP mail server for incoming mail box, type the name of the SMTP mail server. The server name must match the FQDN in the A resource record entry for the mail server in DNS Manager.
- To accept messages only from authenticated users, click Yes for Accept messages from authenticated users only. Otherwise, click No.
- To enable users to create distribution groups from SharePoint sites, click Yes for Allow creation of distribution groups from SharePoint sites. Otherwise, click No.
- Under Distribution group request approval settings, select the actions that will require approval. Actions include the following:
- In the Active Directory container where new distribution groups and contacts will be created box, type the name of the container in the format OU=ContainerName, DC=domain, DC=com, where ContainerName is the name of the OU in AD DS, domain is the second-level domain, and com is the top-level domain.
- Create new distribution group
- Change distribution group e-mail address
- Change distribution group title and description
- Delete distribution group
-
If you want to use a remote Directory Management Service, select Use remote and complete the remainder of this step. Otherwise, click No and proceed to step 8.
If you select this option and you are using Exchange Server, you must configure the DNS Manager and add an SMTP connector. For more information, see Configure DNS Manager and Add an SMTP connector in Microsoft Exchange Server 2010 later in this article. The AD DS has most likely already been configured, so you do not need to do this.
- In the Directory Management Service URL box, type the URL of the Directory Management Service that you want to use. The URL is typically in the following format: http://server:adminport/_vti_bin/SharePointEmailWS.asmx.
- In the SMTP mail server for incoming mail box, type the name of the SMTP mail server. The server name must match the FQDN in the A resource record entry for the mail server in DNS Manager on the domain server.
- To accept messages from authenticated users only, click Yes for Accept messages from authenticated users only. Otherwise, click No.
- To allow creation of distribution groups from SharePoint sites, click Yes for Allow creation of distribution groups from SharePoint sites. Otherwise, click No.
- In the Directory Management Service URL box, type the URL of the Directory Management Service that you want to use. The URL is typically in the following format: http://server:adminport/_vti_bin/SharePointEmailWS.asmx.
- In the Incoming E-Mail Server Display Address section, in the E-mail server display address box, type a display name for the email server (for example, mail.fabrikam.com). You typically use this option together with the Directory Management Service.
Tip:
You can specify the email server address that is displayed when users create an incoming email address for a list or group. Use this setting together with Directory Management Service to provide an email server address that is easy to remember.
-
In the E-Mail Drop Folder section, in the E-mail drop folder box, type the name of the folder from which the Windows SharePoint Services Timer service retrieves incoming email from the SMTP service. This option is available only if you selected Advanced settings mode. If you select this option, ensure that you configure the necessary permissions to the email drop folder. For more information, see Configure permissions to the email drop folder later in this article.
It is useful to have a dedicated email drop folder if the default email drop folder is full or almost full.
Ensure that the logon account for the SharePoint Timer service has Modify permissions on the email drop folder. For more information, see To configure email drop folder permissions for the logon account for the SharePoint Timer service later in this article.
-
In the Safe E-Mail Servers section, select whether you want to accept email from all email servers or from specific email servers.
This option is available only if you selected Automatic settings mode.
- Click OK.
After you configure incoming email, site administrators can configure email–enabled lists and document libraries.
If you selected Directory Management Service, contact addresses that are created for document libraries appear automatically in Active Directory Users and Computers. The addresses are displayed in the OU of AD DS for SharePoint 2013 and must be managed by the administrator of AD DS. The AD DS administrator can add more email addresses for each contact. For more information about AD DS, see Using Active Directory Service in the TechNet Library.
Alternatively, you can configure the computer running Exchange Server by adding a new Exchange Server Global recipient policy. The policy automatically adds external addresses that use the second-level domain name and not the subdomain or host name for SharePoint 2013. For more information about how to manage Exchange Server, see Recipient Configuration Node in the Exchange Server Technical Library.
-
Prepare your environment for incoming email in an advanced scenario
Before you configure incoming email in an advanced scenario, you need to perform additional procedures depending on how you want your incoming email environment to work.
If you want to use Directory Management Service, you must first configure AD DS, and if you use Exchange Server, you must also configure the DNS Manager and add an SMTP connector.
If you want to use a specific email drop folder, ensure that you configure the necessary permissions to the email drop folder.
In this section:
- Configure AD DS to be used with Directory Management Service
- Configure DNS Manager
- Add an SMTP connector in Microsoft Exchange Server 2010
If you plan to use Directory Management Service, you should first create an organizational unit (OU) and make the necessary configurations in AD DS.
To use Directory Management Service on a SharePoint farm, you must configure the application pool identity account for the SharePoint Central Administration website to have the Create, delete, and manage user accounts user right to the container that you specify in AD DS. The preferred way to do this is by assigning the right to the application pool identity account for the SharePoint Central Administration website. An AD DS administrator must set up the OU and assign the Create, delete, and manage user accounts right to the container. The advantage of using Directory Management Service on a remote server farm is that you do not have to assign rights to the OU for multiple farm service accounts.
The following procedures are performed on a domain controller that runs Windows Server 2008 with DNS Manager. In some deployments, these applications might run on multiple servers in the same domain.
To create an OU in AD DS
- Configure AD DS to be used with Directory Management Service
- Verify that the user account that is performing this procedure is a member of the Domain Administrators group or a delegated authority for domain administration on the domain controller that is running DNS Manager.
- Click Start, point to Administrative Tools, and then click Active Directory Users and Computers.
- In Active Directory Users and Computers, right-click the folder for the second-level domain that contains your server farm, point to New, and then click Organizational Unit.
-
Type the name of the OU, and then click OK.
After you create the OU, you must delegate the Create, delete, and manage user accounts right to the container of the OU to manage the user accounts.
To delegate the right to the application pool identity account for Central Administration
- Verify that the user account that is performing this procedure is a member of the Domain Administrators group or the Enterprise Administrators group in AD DS, or a delegated authority for domain administration.
- In Active Directory Users and Computers, find the OU that you created.
- Right-click the OU, and then click Delegate control.
- On the Welcome page of the Delegation of Control Wizard, click Next.
- On the Users and Groups page, click Add, and then type the name of the application pool identity account that the Central Administration uses.
- In the Select Users, Computers, and Groups dialog box, click OK.
- On the Users or Groups page of the Delegation of Control Wizard, click Next.
- On the Tasks to Delegate page of the Delegation of Control Wizard, select the Create, delete, and manage user accounts check box, and then click Next.
- On the last page of the Delegation of Control Wizard, click Finish to exit the wizard.
To create and delete child objects, you must also delegate Create all Child Objects and Delete all Child Objects control of the OU to the application pool identity account for Central Administration. After you complete this procedure, the application pool identity account for Central Administration has Create all Child Objects and Delete all Child Objects control on the OU, and you can enable incoming email.
To delegate Create all Child Objects and Delete all Child Objects control of the OU to the application pool identity account for Central Administration
- Verify that the user account that is performing this procedure is a member of the Domain Administrators group or the Enterprise Administrators group in AD DS, or a delegated authority for domain administration.
- Right-click the OU, and then click Delegate control.
- In the Delegation of Control Wizard, click Next.
- Click Add, and then type the name of the application pool identity account for Central Administration.
- Click OK.
- Click Next.
- On the Tasks to Delegate page of the Delegation of Control Wizard, select Create a custom task to delegate, and then click Next.
- Click This folder, existing objects in this folder, and creation of new objects in this folder, and then click Next.
- In the Permissions section, select Create all Child Objects and Delete all Child Objects.
- Click Next.
- On the last page of the Delegation of Control Wizard, click Finish to exit the wizard.
Delegating Create all Child Objects and Delete all Child Objects control of the OU to the application pool identity account for Central Administration enables administrators to enable email for a list. After these controls have been delegated, administrators cannot disable email for the list or document library because the Central Administration account tries to delete the contact from the whole OU instead of from the list.
To avoid this problem, you must add Delete Subtree permissions for the application pool identity account for Central Administration. Use the following procedure to add these permissions. After this procedure is complete, you can disable incoming email for a list.
To add Delete Subtree permissions for the application pool identity account for Central Administration
- Verify that the user account that is performing this procedure is a member of the Domain Administrators group or the Enterprise Administrators group in AD DS, or a delegated authority for domain administration.
- In Active Directory Users and Computers, click the View menu, and then click Advanced Features.
- Right-click the OU, and then click Properties.
- In the Properties dialog box, click the Security tab, and then click Advanced.
-
In the Permission Entries area, double-click the application pool identity account for Central Administration.
If the application pool identity account is listed more than once, select the first one.
- In the Permissions area, select Allow, for Delete Subtree.
- Click OK to close the Permissions dialog box.
- Click OK to close the Properties dialog box.
- Click OK to close Active Directory Users and Computers.
After you add these permissions, you must restart Internet Information Services (IIS) for the farm.
For more information, see Active Directory Users, Computers, and Groups in the TechNet Library.
If you are using Exchange Server and are routing email internally in your organization, you must create a host (A) resource record in DNS Manager to associate DNS domain names of computers (or hosts) to their IP addresses. Your organization might already have a configured DNS Manager and an A resource record. If not, then use the following procedure.
To create an A resource record for a subdomain
- Verify that the user account that is performing this procedure is a member of the Administrators group on the local computer.
- In DNS Manager, select the forward lookup zone for the domain that contains the subdomain for SharePoint 2013.
- Right-click the zone, and then click New Host (A or AAAA).
- In the New Host dialog box, in the Name text box, type the host or subdomain name for SharePoint 2013.
- In the Fully qualified domain name (FQDN) text box, type the FQDN for the server that is running SharePoint 2013. This is typically in the format subdomain.domain.com.
- Ensure that the domains that are listed under the SMTP server in IIS match the FQDN of the server that receives email. If they do not match, you must create a local domain. For instructions, see To create a local domain later in this article.
- In the IP address text box, type the IP address to which you want the FQDN to resolve.
- Click Add Host.
- In the message that confirms the creation of the host record, click OK.
-
In the New Host dialog box, click Done.
The A resource record now appears in DNS Manager.
If you use the E-mail server display address option and if the email address to which you are sending email messages is not the same as your server name, you must create a local domain.
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) 6.0 Manager.
- In IIS Manager, expand the SMTP server.
- Right-click Domains, and on the Action menu, point to New, and then click Domain.
- In the New SMTP Domain Wizard dialog box, select Alias, and then click Next.
-
In the Domain Name area, in the Name box, type the address of the mail that is to be received by this domain.
This address must be the same as the one that you specified in step 4 in To create an A resource record for a subdomain, and in step 6b in To configure incoming email in an advanced scenario.
- Click Finish.
- In the message that confirms the creation of the host record, click OK.
- Restart the SMTP server so that all email messages that are still in the Queue folder move to the Drop folder. The messages are then sent by the Windows SharePoint Services Timer service to their destination list or library.
Note:
If you are routing email from outside your organization to an SMTP server, you must use an MX record. For more information, see Add a mail exchanger (MX) resource record to a zone in the Windows Server Technical Library.
An SMTP connector gives you more control over the message flow in your organization. Other reasons to use an SMTP connector are to set delivery restrictions or to specify a specific address space. If you use Exchange Server to route incoming email to SharePoint lists and libraries, you must have an SMTP connector so that all mail that is sent to the SharePoint domain uses the servers that are running the SMTP service.
Use the following procedure to add an SMTP connector in Exchange Server. After you complete the procedure, the SMTP connector ensures that incoming email messages are sent to the correct list and library in the farm.
To add an SMTP connector in Exchange Server
- Verify that the user account that is performing this procedure is a member of the Administrators group on the server that is running Exchange Server.
-
In Exchange Management Console, expand the Organization Configuration group, right-click Hub Transport, point to New Send Connector.
The New Send Connector wizard appears.
-
On the Introduction page, do the following and then click Next:
- In the Name box, type a name for the SMTP connector.
- In the Select the intended use for this Send connector box, select the Custom usage type for the connector.
- In the Name box, type a name for the SMTP connector.
- On the Address Space page, click Add, and then click SMTP Address Space.
-
In the SMTP Address Space dialog box, do the following:
- In the Address box, type an email domain for the connector.
- In the Cost box, assign an appropriate cost. By default, the cost is 1.
- In the Address box, type an email domain for the connector.
- Click OK to return to the Address Space page, and then click Next.
- On the Network settings page, select Use domain name system (DNS) “MX” records to route mail automatically, and then click Next.
-
On the Source Server page, click Next.
The Source server page only appears on Hub Transport servers. By default, the Hub Transport server that you are currently working on is listed as a source server.
- On the New Connector page, review your options and then click New to create the new send connector.
-
On the Completion page, ensure that the send connector was created, and then click Finish.
In the Hub Transport pane, you can see that the send connector has been enabled automatically.
For more information, see Create an SMTP Send Connector in the Exchange Server Technical Library.
You can specify a particular email drop folder, which enables SharePoint 2013 to retrieve incoming email from a network share on another server. You can use this option if you do not want to use an SMTP service. However, the drawback of using this option is that SharePoint 2013 cannot detect configuration changes on the remote email server that is delivering email to the drop folder. The result is that SharePoint 2013 cannot retrieve email if the location of the email messages has changed. However, this feature is useful if the default email drop folder is full or almost full.
If you specified an email drop folder, you must ensure that the application pool identity accounts for Central Administration and for the web application have the required permissions to the email drop folder.
-
Configure email drop folder permissions for the application pool identity account for a web application
If your deployment uses different application pool identity accounts for Central Administration and for one or more web applications, each application pool identity account must have permissions to the email drop folder. If the application pool identity account for the web application does not have the required permissions, email will not be delivered to document libraries on that web application.
In most cases, when you configure incoming email and select an email drop folder, permissions are added for the following worker process groups:
- WSS_Admin_WPG, which includes the application pool identity account for Central Administration and the logon account for the SharePoint Timer service, and has Full Control permissions.
- WSS_WPG, which includes the application pool accounts for web applications, and has Read & Execute, List Folder Contents, and Read permissions.
In some cases, these groups might not be configured automatically for the email drop folder. For example, if Central Administration is running as the Network Service account, the groups or accounts that are needed for incoming email will not be added when the email drop folder is created. Check to determine whether these groups have been added automatically to the email drop folder. If the groups have not been added automatically, you can add them or add the specific accounts that are required.
To configure email drop folder permissions for the application pool identity account for a web application
- Verify that the user account that is performing this procedure is a member of the Administrators group on the server that contains the email drop folder.
- In Windows Explorer, right-click the drop folder, click Properties, and then click the Security tab.
- On the Security tab, under the Group or user names box, click Edit.
- In the Permissions for Windows Explorer dialog box, click Add.
-
In the Select Users, Computers, or Groups dialog box, in the Enter the object names to select box, type the name of the worker process group or application pool identity account for the web application, and then click OK.
This account is listed on the Identity tab of the Properties dialog box for the application pool in IIS.
- In the Permissions for User or Group box, next to Modify, select Allow.
- Click OK.
-
Configure email drop folder permissions for the logon account for the SharePoint Timer service
Ensure that the logon account for the Windows SharePoint Services Timer service has Modify permissions on the email drop folder. If the logon account for the service does not have Modify permissions, email–enabled document libraries will receive duplicate email messages.
To configure email drop folder permissions for the logon account for the SharePoint Timer service
- Verify that the user account that is performing this procedure is a member of the Administrators group on the server that contains the email drop folder.
- In Windows Explorer, right-click the drop folder, click Properties, and then click the Security tab.
- On the Security tab, under the Group or user names box, click Edit.
- In the Permissions for Windows Explorer dialog box, click Add.
-
In the Select Users, Computers, or Groups dialog box, in the Enter the object names to select box, type the name of the logon account for the SharePoint Timer service, and then click OK.
This account is listed on the Log On tab of the Properties dialog box for the service in the Services snap-in.
- In the Permissions for User or Group box, next to Modify, select Allow.
- Click OK.
-
Are attachments missing from email messages that are sent to a SharePoint document library?
If attachments are missing from email messages that are sent to a SharePoint document library, it might be because you associated the document library with an email address. When you do this, Directory Management Service may not add the following two attributes to the user associated with the email address:
- internet Encoding = 1310720
- mAPIRecipient = false
You must use Active Directory Service Interfaces (ADSI) to manually add these two missing attributes.
On servers that are running Windows Server 2008 or Windows Server 2008 R2, ADSI Edit is installed when you configure a server as a domain controller by installing the AD DS role. You can also install Windows Server 2008 Remote Server Administration Tools (RSAT) on domain member servers or stand-alone servers. For more information, see Installing or Removing the Remote Server Administration Tools Pack in the Windows Server Technical Library.
To add attributes by using ADSI Edit
- internet Encoding = 1310720
- Click Start, and then click Run.
- In the Run dialog box, type Adsiedit.msc, and then click OK.
- In the ADSI Edit window, expand ADSI Edit, expand Domain [DomainName], expand DC=DomainName, DC=com, and then expand CN=Users.
- Right-click the user name to which you want to add the missing attributes, and then click Properties.
- In the Properties dialog box, double-click Internet Encoding on the Attribute Editor tab.
- In the Integer Attribute Editor dialog box, type 1310720 in the Value box, and then click OK.
- In the Properties dialog box, double-click mAPIRecipient on the Attribute Editor tab.
- In the Boolean Attribute Editor dialog box, click False, and then click OK two times.
-
Configure outgoing email for a SharePoint 2013 farm
Published: July 16, 2012
Summary: Learn how to install and configure the SMTP service and configure outgoing email for a SharePoint 2013 farm.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article describes how to configure outgoing email for a farm or for a specific web application for SharePoint 2013. This article also describes how to install and configure the SMTP service that you must use to enable outgoing email.
After you have installed SharePoint 2013 and completed the initial configuration of your server farm, you can configure outgoing email. Doing so enables users to create alerts to track such site items as lists, libraries, and documents. In addition, site administrators can receive administrative messages about site administrator issues, such as the information that site owners have exceeded their specified storage space. For more information, see Plan outgoing email (SharePoint 2013 Preview).
To configure outgoing email for a specific web application, first configure the default outgoing email for all web applications in the farm. If you configure the outgoing email for a specific web application, that configuration will override the default configuration for all web applications in the farm.
Important:
You cannot configure outgoing email by using Windows PowerShell.
In this article:
- Before you begin
- To install the SMTP service
- To install IIS 6.0 Management tools
- To configure the SMTP service
- To set the SMTP service to start automatically
- To configure outgoing email for a farm by using Central Administration
- To configure outgoing email for a specific web application by using Central Administration
- Before you begin
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
Before you begin this operation, review the following information about prerequisites:
- Your computer is running SharePoint 2013.
- One or more servers in the server farm are running the Simple Mail Transfer Protocol (SMTP) service and have a valid SMTP server address. Alternatively, you must know the name of a server outside the farm that is running the SMTP service.
If you have not installed and configured the SMTP service, before you configure outgoing email you must complete the steps in:
- Install and configure the SMTP service
- Plan browser support
-
Install and configure the SMTP service
Before you can enable outgoing email, you must determine which SMTP server to use. This SMTP server must be configured to allow anonymous SMTP email submissions. The SMTP server can be a server in the farm or outside the farm.
Note:
If your organization does not allow anonymous SMTP email messages to be sent by using Exchange Server, you can use a local SMTP server in the SharePoint farm that accepts anonymous email messages. The local SMTP server automatically authenticates the messages and then forwards them to the computer that‘s running Exchange Server.
-
Install the SMTP service
To install the SMTP service, use the Add Features Wizard in Server Manager. The wizard creates a default SMTP configuration. You can customize this default SMTP configuration to meet the requirements of your organization.
If you already have the SMTP service installed on a server, skip to Configure the SMTP service later in this article.
To install the SMTP service
-
- Verify that the user account that is performing this procedure is a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Server Manager.
- In Server Manager, click Features.
- In Features Summary, click Add Features to open the Add Features Wizard.
- On the Select Features page, select SMTP Server.
- In the Add Features Wizard dialog box, click Add Required Roll Services, and then click Next.
- On the Confirm Installation Selections page, click Install.
- On the Installation Results page, ensure that the installation is complete, and then click Close.
After you install the SMTP service, you configure it to send email messages from servers in the farm.
You can decide to send relayed email messages to all servers except those that you specifically exclude. Alternatively, you can block messages to all servers except those that you specifically include. You can include servers individually or in groups by subnet or domain.
If you enable anonymous access and relayed email messages, you increase the possibility that the SMTP server will be used to relay unsolicited commercial email messages (spam). It is important to limit this possibility by carefully configuring mail servers to help protect against spam. One way that you can do this is by limiting relayed email messages to a list of specific servers or to a domain, and by preventing relayed email messages from all other servers.
Note:
To manage the SMTP service on Windows Server 2008, you must use Internet Information Services (IIS) 6.0 Manager. Ensure that you install IIS 6.0 Management tools in Server Manager.
To install IIS 6.0 Management tools
- Verify that the user account that is performing this procedure is a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Server Manager.
- In Server Manager, click Roles.
- In the Application Server section, click Add Role Services.
- On the Select Role Services page, select Management Tools and IIS 6 Management compatibility, and then click Install.
To configure the SMTP service
- Verify that the user account that is performing this procedure is a member of the Administrators group on the front-end web server.
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) 6.0 Manager.
- In IIS Manager, expand the server name that contains the SMTP server that you want to configure.
- Right-click the SMTP virtual server that you want to configure, and then click Start.
- Right-click the SMTP virtual server that you want to configure, and then click Properties.
- On the Access tab, in the Access control area, click Authentication.
- In the Authentication dialog box, verify that Anonymous access is selected.
- Click OK.
- On the Access tab, in the Relay restrictions area, click Relay.
- To enable relayed email messages to any server, click All except the list below.
-
To accept relayed email messages from one or more specific servers, follow these steps:
- Click Only the list below.
- Click Add, and then add servers one at a time by IP address, or in groups by using a subnet or domain.
- Click OK to close the Computer dialog box.
- Click Only the list below.
- Click OK to close the Relay Restrictions dialog box.
- Click OK to close the Properties dialog box.
Ensure that the SMTP service is running and set to start automatically. To do this, use the following procedure.
To set the SMTP service to start automatically
- Click Start, point to Administrative Tools, and then click Services.
- In Services, right-click Simple Mail Transfer Protocol (SMTP), and then select Properties.
- In the Simple Mail Transfer Protocol (SMTP) Properties dialog box, on the General tab, in the Startup type list, select Automatic.
- Click OK.
-
Configure outgoing email for a farm
You can configure outgoing email for a farm by using the SharePoint Central Administration website. Use the following procedures to configure outgoing email. After you complete the procedures, users can track changes and updates to individual site collections. In addition, site administrators can, for example, receive notices when users request access to a site.
To configure outgoing email for a farm by using Central Administration
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group on the server that is running the SharePoint Central Administration website.
- In Central Administration, click System Settings.
- On the System Settings page, in the E-Mail and Text Messages (SMS) section, click Configure outgoing e-mail settings.
- On the Outgoing E-Mail Settings page, in the Mail Settings section, type the SMTP server name for outgoing email (for example, mail.example.com) in the Outbound SMTP server box.
- In the From address box, type the email address as you want it to be displayed to email recipients.
- In the Reply-to address box, type the email address to which you want email recipients to reply.
- In the Character set list, select the character set that is appropriate for your language.
- Click OK.
-
Configure outgoing email for a specific web application
You can configure outgoing email for a specific web application by using the Central Administration website. Use the following procedures to configure outgoing email.
Note:
To configure outgoing email for a specific web application, first configure the default outgoing email for all web applications in the farm. If you configure the outgoing email for a specific web application, that configuration will override the default configuration for all web applications in the farm.
To configure outgoing email for a specific web application by using Central Administration
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group on the server that is running the SharePoint Central Administration website.
- In Central Administration, in the Application Management section, click Manage web applications.
- On the Web Applications Management page, select a web application, and then in the General Settings group on the ribbon, click Outgoing E-mail.
- On the Web Application Outgoing E-Mail Settings page, in the Mail Settings section, type the name of the SMTP server for outgoing email (for example, mail.fabrikam.com) in the Outbound SMTP server box.
- In the From address box, type the email address (for example, the site administrator alias) as you want it to be displayed to email recipients.
- In the Reply-to address box, type the email address (for example, a help desk alias) to which you want email recipients to reply.
- In the Character set list, click the character set that is appropriate for your language.
- Click OK.
-
Configure services and service applications in SharePoint 2013
Published: July 16, 2012
Summary: Introduces articles that describe how to configure services for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
In SharePoint 2013, you can configure individual services independently, and you can implement only the services that your organization needs.
Deployed services are named service applications. A service application provides a resource that you can share across sites in a farm or sometimes across multiple farms, and users can access them through a hosting web application. Service applications are associated to web applications by service application connections.
For more information about service applications and services, see Technical diagrams (SharePoint 2013 Preview). If you plan to use Office Web Apps, you must install and configure them to work with SharePoint 2013. For more information, see Office Web Apps overview (Used with SharePoint 2013 Preview Products).
The following articles on TechNet and related resources provide information about how to configure services for SharePoint 2013.
-
TechNet articles about how to configure services for SharePoint 2013
The following articles about how to configure services for SharePoint 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Deploy Excel Services to a SharePoint Server 2013 farm by creating an Excel Services service application by using Central Administration.
Configure a Business Intelligence Center in SharePoint Server 2013
Create a Business Intelligence Center in SharePoint Server 2013 by using the Business Intelligence Center enterprise template and creating a new site collection.
Create and configure Machine Translation services in SharePoint Server 2013
The Machine Translation Service in SharePoint 2013 lets users automatically translate documents. This topic describes how to create a Machine Translation Service Application and configure the Machine Translation Service by using Central Administration, or Windows PowerShell.
Configure PerformancePoint Services (SharePoint Server 2013)
Configure PerformancePoint Services in SharePoint Server 2013 before you make it available to users
Create and configure a Search service application in SharePoint Server 2013
This article describes how to create and configure a SharePoint Search service application so that you can crawl content and provide search results to users.
Learn how to retrieve, clone, add, move, remove and activate search components in the search topology using Windows PowerShell. Use these procedures to scale out or scale down the search topology of the Search service application
This article describes how to create a SharePoint Search Center site and grant site access to users.
Learn how to set up SharePoint people search so that users can find people in the organization and the documents that they have authored.
Configure result sources for search in SharePoint Server 2013
Learn how to create and manage result sources for SharePoint Search service applications, and for SharePoint sites and site collections.
This article describes the SharePoint 2013Secure Store Service operations that solution designers can use to create target applications that map user and group credentials to the credentials of external data sources.
Administer the User Profile service in SharePoint Server 2013
Learn how to configure and administer the User Profile service and User Profile Synchronization service.
Visio Graphics Service administration in SharePoint Server 2013
Learn how to create, configure, list, or delete Visio Services service applications by using Central Administration or Windows PowerShell.
Describes the process and cautions that are involved in sharing service applications across farms.
-
Additional resources about how to configure services for SharePoint 2013
The following resources about how to configure services for SharePoint 2013 are available from other subject matter experts.
Content
Description
Installation and Deployment for SharePoint 2013 Resource Center
Visit the Resource Center to access videos, Community Sites, documentation, and more.
-
Configure the Secure Store Service in SharePoint 2013
Published: July 16, 2012
Summary: Configure storage of authorization credentials in Secure Store Service on a SharePoint Server 2013 farm. A video demonstration is included.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article describes how to configure the Secure Store Service on a SharePoint Server 2013 farm. Secure Store has important planning considerations associated with it. Be sure to read Plan the Secure Store service (SharePoint Server 2010) before you begin the procedures in this article.
In this article:
-
Configure Secure Store
To configure Secure Store, you perform the following steps:
- Register a managed account in SharePoint Server 2013 to run the Secure Store application pool.
- Start the Secure Store Service on an application server in the farm.
- Create a Secure Store Service service application.
To run the application pool, you must have a standard domain account. No specific permissions are required for this account. Once the account has been created in Active Directory, follow these steps to register it with SharePoint Server 2013.
To register a managed account
- On the SharePoint Central Administration Web site home page, in the left navigation, click Security.
- On the Security page, in the General Security section, click Configure managed accounts.
- On the Managed Accounts page, click Register Managed Account.
- In the User name box, type the name of the account.
- In the Password box, type the password for the Contoso\ExcelAppPool account.
- If you want SharePoint Server 2013 to handle changing the password for the account, select the Enable automatic password change box and specify the password change parameters that you want to use.
- Click OK.
Once you have configured the registered account, you must start the Secure Store Service on an application server in the farm. Because Secure Store deals with sensitive information, we recommend that you use a separate application server just for the Secure Store Service for better security.
To start the Secure Store Service
- On the Central Administration home page, in the System Settings section, click Manage services on server.
- Above the Service list, click the Server drop-down list, and then click Change Server.
- Select the application server where you want to run the Secure Store Service.
- In the Service list, click Start next to Secure Store Service.
Once the service is started, you must create a Secure Store Service service application. Use the following procedure to create the service application.
To create a Secure Store Service service application
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click New, and then click Secure Store Service.
- In the Service Application Name box, type a name for the service application (for example, Secure Store Service).
- In the Database Server box, type the instance of SQL Server where you want to create the Secure Store database.
Note:
Because the Secure Store database contains sensitive information, we recommend that you deploy the Secure Store database to a different instance of SQL Server from the rest of SharePoint Server 2013.
- Select the Create new application pool option and type a name for the application pool in the text box.
- Select the Configurable option, and, from the drop-down list, select the account for which you created the managed account earlier.
- Click OK.
The Secure Store Service has now been configured. The next step is to generate an encryption key for encrypting the Secure Store database.
-
Work with encryption keys
Before using the Secure Store Service, you must generate an encryption key. The key is used to encrypt and decrypt the credentials that are stored in the Secure Store Service database.
-
Generate an encryption key
The first time that you access the Secure Store service application, your only option is to generate a new encryption key. Once the key has been generated, the rest of the Secure Store functionality becomes available.
To generate a new encryption key
-
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- Click the Secure Store service application.
- In the Key Management group, click Generate New Key.
- On the Generate New Key page, type a pass phrase string in the Pass Phrase box, and type the same string in the Confirm Pass Phrase box. This pass phrase is used to encrypt the Secure Store database.
Important:
A pass phrase string must be at least eight characters and must have at least three of the following four elements:
- Uppercase characters
- Lowercase characters
- Numerals
- Any of the following special characters
“! ” # $ % & ‘ ( ) * + , – . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~
Important:
The pass phrase that you enter is not stored. Make sure that you write this down and store it in a safe place. You must have it to refresh the key, such as when you add a new application server to the server farm.
- Click OK.
For security precautions or as part of regular maintenance you may decide to generate a new encryption key and force the Secure Store Service to be re-encrypted based on the new key. You can use this same procedure to do this.
Caution:
You should back up the database of the Secure Store Service application before generating a new key.
Refreshing the encryption key propagates the key to all the application servers in the farm. You may be required to refresh the encryption key if any of the following things are true:
- You add a new application server to the server farm.
- You restore a previously backed up Secure Store Service database and have since changed the encryption key.
- You receive an “Unable to get master key” error message.
- You have upgraded your farm from SharePoint Server 2010.
To refresh the encryption key
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- Click the Secure Store service application.
- In the Key Management group, click Refresh Key.
-
In the Pass Phrase box, type the pass phrase that you first used to generate the encryption key.
This phrase is either the pass phrase that you used when you initialized the Secure Store Service service application or one that you used when you created a new key by using the Generate a New Key command.
- Click OK.
-
Store credentials in Secure Store
Storing credentials in Secure Store is accomplished by using a Secure Store target application. A target application maps the credentials of a user, group, or claim to a set of encrypted credentials stored in the Secure Store database. After a target application is created, you can associate it with an external content type or application model, or use it with a business intelligence service application such as Excel Services or Visio Services to provide access to an external data source. When a SharePoint Server 2013 service application calls the target application, Secure Store confirms that the user making the request is an authorized user of the target application and then retrieves the encrypted credentials. The credentials are then used on the user’s behalf by the SharePoint Server 2013 service application.
To create a target application, you must do the following:
- Create the target application itself, specifying the type of credentials that you want to store in the Secure Store database, the administrators for the target application, and the credential owners.
- Specify the credentials that you want to store.
Target applications are configured on the Secure Store Service Application page in Central Administration. Use the following procedure to create a target application.
To create a target application
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- Click the Secure Store service application.
- In the Manage Target Applications group, click New.
-
In the Target Application ID box, type a text string.
This is the unique string that you will use externally to identify this target application.
- In the Display Name box, type a text string that will be used to display the identifier of the target application in the user interface.
-
In the Contact Email box, type the e-mail address of the primary contact for this target application.
This can be any legitimate e-mail address and does not have to be the identity of an administrator of the Secure Store Service application.
- When you create a target application of type Individual (see below), you can implement a custom Web page that lets users add individual credentials for the destination data source. This requires custom code to pass the credentials to the target application. If you did this, type the full URL of this page in the Target Application Page URL field. There are three options:
- Use default page: Any Web sites that use the target application to access external data will have an individual sign-up page that was added automatically. The URL of this page will be http:/<samplesite>/_layouts/SecureStoreSetCredentials.aspx?TargetAppId=<TargetApplicationID>, where <TargetApplicationID> is the string typed in the Target Application ID box. By publicizing the location of this page, you can enable users to add their credentials for the external data source.
- Use custom page: You provide a custom Web page that lets users provide individual credentials. Type the URL of the custom page in this field.
- None: There is no sign-up page. Individual credentials are added only by a Secure Store Service administrator who is using the Secure Store Service application.
- In the Target Application Type drop-down list, choose the target application type: Group, for group credentials, or Individual, if each user is to be mapped to a unique set of credentials on the external data source.
Note:
There are two primary types for creating a target application:
- Group, for mapping all the members of one or more groups to a single set of credentials on the external data source.
- Individual, for mapping each user to a unique set of credentials on the external data source.
- Click Next.
-
Use the Specify the credential fields for your Secure Store Target Application page to configure the various fields which may be required to provide credentials to the external data source. By default, two fields are listed: Windows User Name and Windows Password.
To add an additional field for supplying credentials to the external data source, on the Specify the credential fields for your Secure Store Target Application page, click Add Field.
By default, the type of the new field is Generic. The following field types are available:
|
Description |
|
Generic |
Values that do not fit in any of the other categories. |
|
User Name |
A user account that identifies the user. |
|
Password |
A secret word or phrase. |
|
PIN |
A personal identification number. |
|
Key |
A parameter that determines the functional output of a cryptographic algorithm or cipher. |
|
Windows User Name |
A Windows user account that identifies the user. |
|
Windows Password |
A secret word or phrase for a Windows account. |
|
Certificate |
A certificate. |
|
Certificate Password |
The password for the certificate. |
-
To change the type of a new or existing field, click the arrow that appears next to the type of the field, and then select the new type of field.
Note:
Every field that you add will be required to have data when you set the credentials for this target application.
- You can change the name that a user sees when interacting with a field. In the Field Name column of the Specify the credential fields for your Secure Store Target Application page, change a field name by selecting the current text and typing new text.
- When a field is masked, each character that a user types is not displayed but is replaced with a mask character such as the asterisk “*”. To mask a field, click the check box for that field in the Masked column of the page.
-
To delete a field, click the delete icon for that field in the Delete column of the page.
When you have finished editing the credential fields, click Next.
- In the Specify the membership settings page, in the Target Application Administrators Field, list all users who have access to manage the target application settings.
- If the target application type is group, in the Members field, list the user groups to map to a set of credentials for this target application.
- Click OK to complete configuring the target application.
-
Set credentials for a target application
After creating a target application, an administrator of that target application can set credentials for it. These credentials are used by the calling application to provide access to an external data source. If the target application is of type Individual, you can also enable users to supply their own credentials.
To set credentials for a target application
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- Click the Secure Store service application.
-
In the target application list, point at the target application for which you want to set credentials, click the arrow that appears, and then, in the menu, click Set credentials.
If the target application is of type Group, type the credentials for the external data source. Depending on the information that is required by the external data source, the fields for setting credentials will vary.
If the target application is of type Individual, type the user name of the individual who will be mapped to this set of credentials on the external data source, and type the credentials for the external data source. Depending on the information that is required by the external data source, the fields for setting credentials will vary.
- Click OK.
Once you have set the credentials for the target application, it is ready to be used by a SharePoint Server 2013 service such as Business Connectivity Services or Excel Services.
-
Enable the audit log
Audit entries for the Secure Store service are stored in the Secure Store Service database. By default, the audit log file is disabled.
An audit log entry stores information about a Secure Store Service action, such as when it was performed, whether it succeeded, why it failed if it didn’t succeed, the Secure Store Service user who performed it, and optionally the Secure Store Service user on whose behalf it was performed. Therefore, a valid reason to enable an audit log file is to troubleshoot an authentication issue.
To enable the audit log by using Central Administration
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- Select the Secure Store service application. (That is, select the service application, but do not click the link to go to the Secure Store Service application settings page.)
- On the ribbon, click Properties.
- From the Enable Audit section, click to select the Audit log enabled box.
- To change the number of days that entries will be purged from the audit log file, specify a number in days in the Days Until Purge field. The default value is 30 days.
- Click OK.
-
Video demonstration
This video shows the steps necessary to configure a Secure Store service application.
Note:
This video uses SharePoint Server 2010. Target applications function in the same way in SharePoint Server 2013.
Video: How target applications are used in Secure Store
-
Create and configure a Search service application in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to create and configure a SharePoint Search service application so that you can crawl content and provide search results to users.
Applies to: SharePoint Server 2013
-
Before you begin
If you used the Farm Configuration Wizard after you installed SharePoint Server 2013, a Search service application might have been created at that time. To verify whether a Search service application exists, you can click Manage service applications in the Application Management section on the Central Administration home page. For the remainder of this article, it is assumed that a Search service application does not exist yet, and that therefore you must create one.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
How to create and configure a SharePoint Search service application
When you deploy and configure a Search service application, you perform the following main tasks:
- Create accounts — Certain domain user accounts are required specifically for a Search service application.
- Create a Search service application — A Search service application provides enterprise search features and functionality.
- Configure the Search service application — Basic configuration of a Search service application includes configuring a default content access account, an email contact, and content sources.
- Configure the Search service application topology — You can deploy search components on different servers in the farm. You can also specify which instance of SQL Server is used to host the search-related databases.
-
Step 1: Create accounts that are required for a SharePoint Search service application
The following table lists the accounts that are required when a Search service application is created.
Account
Description
Notes
Search service
Windows user credentials for the SharePoint Server Search service, which is a Windows service
This setting applies to all Search service applications in the farm. You can change this account at any time by clicking Configure service accounts in the Security section on the Central Administration home page.
Search Admin Web Service application pool
- Search Query and Site Settings Web Service application pool
Windows user credentials
For each of these accounts, you can use the same credentials that you specified for the Search service. Or, you can assign different credentials to each account according to the principle of least-privilege administration.
Default content access
Windows user credentials for the Search service application to use to access content when crawling
We recommend that you specify a separate account for the default content access account according to the principle of least-privilege administration.
The accounts that you use for the Search service, the Search Admin Web Service application pool, and the Search Query and Site Settings Web Service application pool must be registered as managed accounts in SharePoint Server 2013 so that they are available when you create the Search service application. Use the following procedure to register each of these accounts as a managed account.
To register a managed account
- On the Central Administration home page, in the Quick Launch, click Security.
- On the Security page, in the General Security section, click Configure managed accounts.
- On the Managed Accounts page, click Register Managed Account.
- On the Register Managed Account page, in the Account Registration section, type the user name and password that you want to use as credentials for the service account.
- If you want SharePoint Server 2013 to manage password changes for this account, select the Enable automatic password change check box and configure the parameters for automatic password change.
- Click OK.
-
Step 2: Create a SharePoint Search service application
Each Search service application has a separate content index. You can create multiple Search service applications if you want to have different content indexes for different sets of content. For example, if you want to segregate sensitive content (such as employee benefits information) into a separate content index, you can create a separate Search service application to correspond to that set of content.
Use the following procedure to create a Search service application.
To create a Search service application
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group for the farm for which you want to create the service application.
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, on the ribbon, click New, and then click Search Service Application.
-
On the Create New Search Service Application page, do the following:
- Accept the default value for Service Application name, or type a new name for the Search service application.
- In the Search Service Account list, select the managed account that you registered in the previous procedure to run the Search service.
-
In the Application Pool for Search Admin Web Service section, do the following:
- Select the Create new application pool option, and then specify a name for the application pool in the Application pool name text box.
- In the Select a security account for this application pool section, select the Configurable option, and then from the list select the account that you registered to run the application pool for the Search Admin Web Service.
- Select the Create new application pool option, and then specify a name for the application pool in the Application pool name text box.
-
In the Application Pool for Search Query and Site Settings Web Service section, do the following:
- Choose the Create new application pool option, and then specify a name for the application pool in the Application pool name text box.
- In the Select a security account for this application pool section, select the Configurable option, and then from the list select the account that you registered to run the application pool for the Search Query and Site Settings Web Service.
- Choose the Create new application pool option, and then specify a name for the application pool in the Application pool name text box.
- Accept the default value for Service Application name, or type a new name for the Search service application.
- Click OK.
-
Step 3: Configure the SharePoint Search service application
You configure a Search service application on the Search Administration page for that service application. Use the following procedure to go to the Search Administration page for a particular Search service application.
To go to the Search Administration page
- Verify that the user account that is performing this procedure is an administrator for the Search service application that you want to configure.
- On the home page of the Central Administration website, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click the Search service application that you want to configure.
On the Search Administration page, configure the settings as described in the following sections:
- Specify the default content access account
- Specify the contact email address
-
Create content sources
-
Specify the default content access account
-
When you create a Search service application, the account that you specify for the Search service is automatically configured as the default content access account. The crawler uses this account to crawl content that does not have an associated crawl rule that specifies a different account. For the default content access account, we recommend that you specify a domain user account that has read access to as much of the content that you want to crawl as possible. You can change the default content access account at any time.
If you have to crawl certain content by using a different account, you can create a crawl rule and specify a different account for crawling. For information about how to create a crawl rule, see Manage crawl rules (SharePoint Server 2013 Preview).
Use the following procedure to specify the default content access account.
To specify the default content access account
- On the Search Administration page, in the System Status section, click the link in the Default content access account row.
- In the Default Content Access Account dialog box, in the Account box, type the account that you created for content access in the form domain\user name.
- Type the password for this account in the Password and Confirm Password boxes.
- Click OK.
-
Specify the contact email address
The Search service writes the contact email address to the logs of crawled servers. The default contact email address, someone@example.com, is a placeholder. We recommend that you change this to an account that an external administrator can contact when a crawl might be contributing to a problem such as a decrease in performance on a server that the search system is crawling.
Use the following procedure to specify the contact email address.
To specify the contact email address
- On the Search Administration page, in the System Status section, click the link for the Contact e-mail address.
- In the Search E-mail Setting dialog box, in the E-mail Address box, type the email address that you want to appear in the logs of servers that are crawled by the search system.
- Click OK.
-
Create content sources in a SharePoint Search service application
Crawling requires at least one content source. A content source is a set of options that you use to specify the type of content to crawl, the starting URLs to crawl, and when and how deep to crawl. When a Search service application is created, a content source named “Local SharePoint sites” is automatically created and configured for crawling all SharePoint sites in the local server farm. You can create content sources to specify other content to crawl and how the system will crawl that content. For more information, see Add, edit, or delete a content source (SharePoint Server 2013 Preview). However, you do not have to create other content sources if you do not want to crawl content other than the SharePoint sites in the local farm.
If you choose the Standalone installation option when you install SharePoint Server 2013, a full crawl of all SharePoint sites in the farm is automatically performed after installation and an incremental crawl is scheduled to occur every 20 minutes after that. If you choose the Server Farm installation option when you install SharePoint Server 2013, no crawls are automatically scheduled or performed.
-
Step 4: Configure the SharePoint Search service application topology
When you create a Search service application, the SharePoint Server Search service is started on the application server that is hosting the Central Administration website, and search components are deployed to that server. If you have more than one application server in your farm, you can deploy additional search components on other application servers, depending on your requirements. You can deploy multiple instances of certain components. For more information, see Manage search topology (SharePoint Server 2013 Preview).
-
Create a Search Center site in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to create a SharePoint Search Center site and grant site access to users.
Applies to: SharePoint Server 2013
Note:
The article does not apply to SharePoint Online, because in that environment a Search Center site is automatically available at <host_name>/search/.
A Search Center site, or Search Center, provides an interface for users to submit search queries and view search results. A Search Center site is the top-level site of a site collection that a farm administrator creates by using the Enterprise Search Center template or the Basic Search Center template.
-
Before you begin
Depending on the kind of installation that you performed and the site collection template that you selected at that time, the farm might already have a Search Center site. To check this, browse to the top-level site for the site collection that you created during installation. In either case, you can create a Search Center site and grant users access to it by using the procedures in this article. After you create the Search Center site, the site collection administrator or site owner might want to add features and functionality so that the site provides a richer interface than the search box that appears by default on each SharePoint site.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
To create a SharePoint Search Center site
- Plan browser support
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group.
- On the home page of the Central Administration website, in the Application Management section, click Create site collections.
-
On the Create Site Collection page, do the following:
- In the Web Application section, select a web application to contain the new site collection. To use a web application other than the one that is displayed, click the web application that is displayed, and then click Change Web Application.
- In the Title and Description section, in the Title box, type the name for the new Search Center site. Optionally, type a description in the Description box.
-
In the Web Site Address section, for the part of the URL immediately after the web application address, select /sites/, or select a managed path that was previously defined, and then type the final part of the URL.
Note the address of the new Search Center for future reference.
-
In the Template Selection section, do the following:
- In the Select the experience version drop-down list, select 2013 to create a Search Center site that provides the SharePoint Server 2013 user experience, or select 2010 to create a Search Center site that provides the SharePoint 2010 Products user experience.
- In the Select a template subsection, click the Enterprise tab, and then do one of the following:
- In the Select the experience version drop-down list, select 2013 to create a Search Center site that provides the SharePoint Server 2013 user experience, or select 2010 to create a Search Center site that provides the SharePoint 2010 Products user experience.
- In the Web Application section, select a web application to contain the new site collection. To use a web application other than the one that is displayed, click the web application that is displayed, and then click Change Web Application.
- If you are using SharePoint Foundation 2013, select the Basic Search Center template.
- Otherwise, if you are using SharePoint Server 2013, select the Enterprise Search Center template.
- In the Primary Site Collection Administrator section, in the User name box, type the user name of the primary site collection administrator for this site collection in the form domain\user name.
- (Optional) In the Secondary Site Collection Administrator section, type the user name of a secondary site collection administrator in the form domain\user name.
-
In the Quota Template section, select No Quota.
A Search Center site is not intended to be a data repository. Therefore, you do not have to select a quota template.
- Click OK.
- On the Top-Level Site Successfully Created page, click the link to the Search Center site that you created.
After you create the Search Center site, you must grant site access to users so that they can perform search queries and view search results. Use the following procedure to grant site access to users.
To grant access to the SharePoint Search Center
- Verify that the user account that is performing this procedure is a member of the Owners group on the Search Center site.
- In a web browser, go to the Search Center site.
- Open the Site menu by clicking the gear icon in the upper-right portion of the page, and then click Site Permissions.
- In the Shared with dialog box, click Invite people.
-
In the Share <SearchCenterName> dialog box, in the Enter users separated with semicolons text box, type the names of the Windows user groups and Windows users to whom you want to grant permissions for submitting queries and viewing search results in the Search Center.
For example, to grant access to the Search Center to all Windows users, type NT Authority\authenticated users.
- Click Show options.
- Clear the Send an email invitation check box.
- In the Select a group or permission level drop-down list, select <SearchCenterName> Visitors [Read].
- Click Share.
-
Deploy people search in SharePoint Server 2013
Published: October 16, 2012
Summary: Learn how to set up SharePoint people search so that users can find people in the organization and the documents that they have authored.
Applies to: SharePoint Server 2013
In this article:
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint 2013
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
-
People search prerequisites
People search has the following prerequisites:
- A Search service application must be running in the farm. For more information, see Create and configure a Search service application in SharePoint Server 2013. The farm must also have a Search Center that uses the Enterprise Search Center template. For more information, see Create a Search Center site in SharePoint Server 2013.
- A Managed Metadata service application must be running in the farm. For more information, see Overview of managed metadata service applications in SharePoint Server 2013.
-
User profile synchronization must be configured in the farm. If this has not been done yet, at a minimum you must complete the following procedures that are described in Synchronize user and group profiles in SharePoint Server 2013:
For more information, see Overview of profile synchronization in SharePoint Server 2013 and Plan profile synchronization for SharePoint Server 2013.
The following sections describe how to deploy and provide data for people search.
- Plan browser support
-
Set up people search
To set up people search, you must configure My Sites settings and configure crawling.
-
Configure My Sites settings
You configure My Sites for a User Profile service application to specify the My Site Host location and other settings. For more information, see Plan for My Sites in SharePoint Server 2013 and Configure My Site settings for the User Profile service application.
After you configure My Sites settings, the next step is to configure crawling.
-
Configure crawling
When you configure My Sites, the default content access account for search is automatically given Retrieve People Data for Search Crawlers permissions in the User Profile service application. If you want to use a different content access account to crawl the profile store, you must make sure that the account has permissions to crawl the profile store. Use the following procedure to grant access to the profile store for a different account.
To grant access to an account to crawl the profile store
-
- Verify that the user account that is performing this procedure is an administrator for the Search service application.
- Start SharePoint 2013 Central Administration.
-
For Windows Server 2008 R2:
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Central Administration.
If SharePoint 2013 Central Administration is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Central Administration.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click the row that contains the User Profile service application, and then in the ribbon, click Administrators.
- In the Administrators for User Profile Service Application dialog box, in the To add an account box, type a user account in the form domain\user name.
- Click Add.
- In the Permissions list, select the Retrieve People Data for Search Crawlers check box.
- Click OK.
After you give the account access to crawl the profile store, you must create a crawl rule to specify that you want to use that account when you crawl the profile store. Use the following procedure to create a crawl rule for this purpose.
To create a crawl rule to authenticate to the User Profile service application
- Verify that the user account that is performing this procedure is an administrator for the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click the Search service application for which you want to create a crawl rule.
- On the Search Administration page, in the Quick Launch, in the Crawling section, click Crawl Rules.
- On the Manage Crawl Rules page, click New Crawl Rule.
- In the Path section, in the Path box, type the start address for the User Profile service application in the form sps3://<hostname>, where <hostname> is the URL for the Web application where you deployed the My Sites site collection.
- Click Use regular expression syntax for matching this rule if you want to use regular expression syntax in the path.
- In the Crawl Configuration section, select Include all items in this path.
- In the Specify Authentication section, select Specify a different content access account.
- In the Account box that appears, type the user account to which you gave access to the profile store in the form domain\user name.
- Type the password for the account that you specified in the Password and Confirm Password boxes.
- Clear the Do not allow Basic Authentication check box only if you want to allow the user account credentials to be sent as plaintext.
Note:
You should not clear the Do not allow Basic Authentication check box unless you are using SSL to encrypt the website traffic. For more information, see Plan for user authentication methods in SharePoint 2013.
- Click OK.
For more information, see Manage crawl rules.
When you configure My Sites, the starting URL to crawl the profile store (sps3://<hostname>) is automatically added to the default content source. We recommend that you remove the URL of the profile store from the default content source and then create a separate content source to crawl only the profile store. This allows you to crawl the profile store on a different schedule from other crawls.
Use the following procedure to remove the URL of the profile store from the default content source.
To remove the profile store URL from the default content source
- Verify that the user account that is performing this procedure is an administrator for the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click Search Service Application.
- On the Search Administration page, in the Quick Launch, in the Crawling section, click Content Sources.
- On the Manage Content Sources page, click the link to the default content source (Local SharePoint sites).
- In the Start Addresses section, remove the URL for the profile store (sps3://<hostname>, where <hostname> is the URL for the web application where you deployed the My Sites site collection).
-
Click OK.
Use the following procedure to create a content source that specifies how to crawl the profile store.
To create a content source that specifies how to crawl the profile store
- Verify that the user account that is performing this procedure is an administrator for the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click Search Service Application.
- On the Search Administration page, in the Quick Launch, in the Crawling section, click Content Sources.
- On the Manage Content Sources page, click New Content Source.
- On the Add Content Source page, in the Name section, type a name for this content source.
- In the Content Source Type section, ensure that SharePoint Sites is selected.
- In the Start Addresses section, type the start address in the form sps3://<hostname>, where <hostname> is the URL for the web application where you deployed the My Sites site collection.
- In the Crawl Settings section, leave the default value of Crawl everything under the host name for each start address.
- In the Crawl Schedules section, do the following:
-
Select Enable Continuous Crawls or Enable Incremental Crawls.
A continuous crawl automatically provides maximum freshness for the content source without an incremental crawl schedule. For more information, see Manage continuous crawls in SharePoint 2013.
If you select Enable Incremental Crawls, create an incremental crawl schedule.
- Optionally create a schedule for full crawls.
- If you selected Enable Incremental Crawls, in the Content Source Priority section, select the priority for this content source.
Note:
The Content Source Priority section does not appear when you specify the content source type as SharePoint Sites and you select Enable Continuous Crawls.
- Click OK.
-
Add data for people search
To get the best results from people search, you should add as much information as you can by adding user profiles to the profile store and adding information to My Sites.
-
Add user profiles to the profile store
Before you can obtain meaningful people search results, you must add user profiles to the User Profile service application. You can do this in the following ways:
-
Import user profiles from the directory service. For more information, see the following articles:
- Plan for profile synchronization (SharePoint 2013 Preview)
- Synchronize user and group profiles in SharePoint Server 2013
- Configure profile synchronization by using SharePoint Active Directory Import in SharePoint Server 2013
- Configure profile synchronization using a Lightweight Directory Interchange Format (LDIF) file in SharePoint 2013
- Plan for profile synchronization (SharePoint 2013 Preview)
- Add user profiles manually.
- Synchronize with an external data source by using the Business Data Connectivity service. For more information, see Phase 3: Configure connections and import data from business systems in the article Synchronize user and group profiles in SharePoint Server 2013.
Important:
For a test environment, we recommend that you do not synchronize the profile store to a directory service or other external data source that is in a production environment. Instead, create a copy of the directory service and synchronize the copy with the profile store.
Use the following procedure to view the user profiles in the User Profile service application.
To view a list of user profiles in the User Profile service application
-
- Verify that the user account that is performing this procedure is an administrator for the User Profile service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click the User Profile service application.
- On the Manage Profile Service page, in the People section, click Manage User Profiles.
-
On the Manage User Profiles page, in the Find profiles box, type the name of the domain of which the users are members.
Do not type the fully qualified domain name. For example, if users are members of the Contoso.com domain, type Contoso in the Find profiles box.
- Click Find.
-
Add information to My Sites
My Sites keep information in the User Profile service application databases. The User Profile service application stores much of the information that appears in results for people search. People search results become more useful as users add more information to their My Sites.
The first time that a user accesses their My Site, also known as their personal site, a My Site is created for them and a profile is automatically added to the User Profile service application.
To add information to a user’s My Site, log on as a user for whom a user profile was created in the User Profile service application, and then go to that user‘s My Site. In the user‘s My Site, you can provide information about the user‘s expertise and interests. To see how the information that you added affects the people search results that appear, perform a crawl of the profile store, and then search on the user’s name.
-
Crawl the profile store
You are now ready to crawl the profile store. For information about how to start the crawl, see Start, pause, resume, or stop crawls in SharePoint 2013.
Note:
We recommend that you crawl the profile store and wait about two hours after the crawl finishes before you start the first crawl of the default content source (that is, local SharePoint sites). After the crawl of the profile store finishes, the search system generates a list to standardize people’s names. This is so that when a person’s name has different forms in search results, the results are displayed in a single group corresponding to one name. For example, all documents authored by Anne Weiler or A. Weiler or alias AnneW can be displayed in the search results in a result block that is labeled “Documents by Anne Weiler”. Similarly, all documents authored by any of those identities can be displayed under the heading “Anne Weiler” in the refinement panel if “Author” is one of the categories there.
For information about how to view the status of a crawl, see Start, pause, resume, or stop a crawl.
-
Configure result sources for search in SharePoint Server 2013
Published: October 16, 2012
Summary: Learn how to create and manage result sources for SharePoint Search service applications, and for SharePoint sites and site collections.
Applies to: SharePoint Server 2013
Result sources limit searches to certain content or to a subset of search results. SharePoint Server 2013 provides 16 pre-defined result sources. The pre-configured default result source is Local SharePoint Results. You can specify a different result source as the default. For more information, see Understanding result sources.
In this article:
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Create a result source
You can create a result source for a Search service application, a site collection, or a site. The following table shows the permissions that are required to create a result source at each level, and where the result source can be used.
-
Levels and permissions for result sources
When you create a result source at this level
You must have this permission
The result source can be used in
Search service application
Search service application administrator
All site collections in web applications that consume the Search service application
Site collection
Site collection administrator
All sites in the site collection
Site
Site owner
The site
To create a result source
-
- Depending on the level at which you want to create the result source, do one of the following:
- To create a result source for a Search service application:
- Verify that the user account that performs this procedure is an administrator on the Search service application.
- In Central Administration, in the Application Management section, click Manage service application.
- Click the Search service application for which you want to create a result source.
- On the Search Administration page for the Search service application, on the Quick Launch, in the Queries and Results section, click Result Sources.
- To create a result source for a site collection:
- Verify that the user account that performs this procedure is an administrator for the site collection.
- On the Settings menu for the site collection, click Site Settings.
- On the Site Settings page, in the Site Collection Administration section, click Search Result Sources.
- To create a result source for a site:
- Verify that the user account that performs this procedure is a member of the Owners group for the site.
- On the Settings menu for the site, click Site Settings.
- On the Site Settings page, in the Search section, click Result Sources.
- On the Manage Result Sources page, click New Result Source.
-
On the Add Result Source page, in the General Information section, do the following:
- In the Name box, type a name for the result source.
- In the Description box, type a description of the result source.
- In the Name box, type a name for the result source.
- In the Protocol section, select one of the following protocols for retrieving search results:
- Local SharePoint, the default protocol, provides results from the search index for this Search service application.
- Remote SharePoint provides results from the index of a search service in another farm.
- OpenSearch provides results from a search engine that uses the OpenSearch 1.0/1.1 protocol.
-
Exchange provides results from Microsoft Exchange Server. Click Use AutoDiscover to have the search system find an Exchange Server endpoint automatically, or type the URL of the Exchange web service to retrieve results from — for example, https://contoso.com/ews/exchange.asmx.
Note:
Note: The Exchange Web Services Managed API must be installed on the computer on which the search service is running. For more information, see Optional software in Hardware and software requirements for SharePoint 2013.
- In the Type section, select SharePoint Search Results to search the whole index, or People Search Results to enable query processing that is specific to people search.
- In the Query Transform field, do one of the following:
- Leave the default query transform (searchTerms) as is. In this case, the query will be unchanged since the previous transform.
- Type a different query transform in the text box.
- Use the Query Builder to configure a query transform by doing the following:
- Click Launch Query Builder.
- In the Build Your Query dialog box, optionally build the query by specifying filters, sorting, and testing on the tabs as shown in the following tables.
-
On the BASICS tab
|
Keyword filter |
You can use keyword filters to add pre-defined query variables to the query transform. You can select pre-defined query variables from the drop-down list, and then add them to the query by clicking Add keyword filter. |
|
Property filter |
You can use property filters to query the content of managed properties that are set to queryable in the search schema. You can select managed properties from the Property filter drop-down list. Click Add property filter to add the filter to the query. |
-
On the SORTING tab
|
Sort results |
In the Sort by menu, you can select a managed property from the list of managed properties that are set as sortable in the search schema, and then select Descending or Ascending. To sort by relevance, that is, to use a ranking model, select Rank. You can click Add sort level to specify a property for a secondary level of sorting for search results. |
|
Ranking Model |
If you selected Rank from the Sort by list, you can select the ranking model to use for sorting. |
|
Dynamic ordering |
You can click Add dynamic ordering rule to specify additional ranking by adding rules that change the order of results within the result block when certain conditions are satisfied. |
-
On the TEST tab
|
Query text |
You can view the final query text, which is based on the original query template, the applicable query rules, and the variable values. |
|
Click Show more to display the options in the following rows of this table. |
|
|
Query template |
You can view the query as it is defined in the BASICS tab or in the text box in the Query transform section on the Add Result Source page. |
|
Query template variables |
You can test the query template by specifying values for the query variables. |
- On the Add Result Source page, in the Credentials Information section, select the authentication type that you want for users to connect to the result source.
-
Set a result source as default
You can set any result source as the default result source. Specifying a result source as default can make it easier to edit the query in Search Web Parts. For example, when you add a Content Search Web Part to a page, the Web Part automatically uses the default result source. For more information, see Configure Search Web Parts in SharePoint Server 2013.
To set a result source as default
- Perform the appropriate procedures in the following list depending on the level at which the result source was configured.
- If the result source was created at the Search service application level, do the following:
- Verify that the user account that performs this procedure is an administrator for the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- Click the Search service application for which you want to set the result source as default.
- On the Search Administration page, in the Queries and Results section, click Result Sources.
- If the result source is at the site collection level, do the following:
- Verify that the user account that performs this procedure is an administrator for the site collection administrator.
- On the Settings menu for the site collection, click Site Settings.
- On the Site Settings page, in the Site Collection Administration section, click Search Result Sources.
- If the result source is at the site level, do the following:
- Verify that the user account that performs this procedure is a member of the Owners group for the site.
- On the Settings menu for the site, click Site Settings.
- On the Site Settings page, in the Search section, click Result Sources.
- On the Manage Result Sources page, point to the result source that you want to set as default, click the arrow that appears, and then click Set as Default.
-
Create and configure Machine Translation services in SharePoint Server 2013
Updated: October 16, 2012
Summary: Learn how to create a new SharePoint Machine Translation service application and how to configure the Machine Translation Service.
Applies to: SharePoint Server 2013
The Machine Translation Service in SharePoint Server 2013 lets users automatically translate documents. You can create a Machine Translation service application and configure the Machine Translation Service by using Central Administration, or Windows PowerShell. Configuring the Machine Translation Service consists of the following steps:
- Create a Machine Translation service application.
- Configure the Machine Translation Service.
-
Before you begin
Before you perform these operations, review the following information about prerequisites:
- The App Management service application must be started in Central Administration. For more information, see Configure an environment for apps for SharePoint 2013.
- You must configure server-to-server authentication and app authentication. For more information, see Configure server-to-server authentication in SharePoint 2013 and Configure app authentication in SharePoint Server 2013.
- There must be a User Profile service application proxy in the default proxy group for the farm, and the User Profile service application must be started and configured by using Central Administration or by using Windows PowerShell. For more information, see Create, edit, or delete a User Profile service application (SharePoint 2013 Preview).
- The server from which machine translations will be run must be able to connect to the Internet.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support (http://go.microsoft.com/fwlink/p/?LinkId=246502)
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products (http://go.microsoft.com/fwlink/p/?LinkId=246501)
- Keyboard shortcuts (http://go.microsoft.com/fwlink/p/?LinkID=246504)
- Touch (http://go.microsoft.com/fwlink/p/?LinkId=246506)
- The App Management service application must be started in Central Administration. For more information, see Configure an environment for apps for SharePoint 2013.
-
Create a SharePoint Machine Translation service application
You can create a new Machine Translation Service application by using either Central Administration or Windows PowerShell.
To create a Machine Translation service application by using Central Administration
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group and the Administrators group on the computer that is running Central Administration.
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- On the ribbon, click New, and then click Machine Translation Service.
- In the Create New Machine Translation Service Application pane, in the Name section, type a name for the service application.
- In the Application Pool section, do one of the following:
- Click Use existing application pool, and then select the application pool that you want to use from the drop-down list.
-
Click Create a new application pool, type the name of the new application pool, and then under Select a security account for this application pool do one of the following:
- Click Predefined to use a predefined security account, and then select the security account from the drop-down list.
- Click Configurable to specify a new security account to be used for an existing application pool. You can create a new account by clicking the Register new managed account link.
- Click Predefined to use a predefined security account, and then select the security account from the drop-down list.
Important:
The account that is used by the application pool must also have Full Control permissions to the User Profile service application. If you create a new application pool and a new account, make sure that you add the account to the list of accounts that can use the User Profile Service Application, and grant Full Control permissions to the account. For more information, see Restrict or enable access to a service application (SharePoint Server 2010).
- In the Partitioned Mode section, select Run in partitioned mode only if you will be providing hosting services for other sites, and the sites using it have site subscriptions.
- In the Add to Default Proxy List section, select Add this service application’s proxy to the farm’s default proxy list. If you have multiple Web applications, and want them to use different sets of services, clear this check box.
- In the Database section, specify the database server, database name, and authentication method for the new service application as described in the following table. The database is used to hold the work items for the Machine Translation service.
-
Database section properties
|
Item |
Action |
|
Database Server |
Type the name of the database server and SQL Server 2012 instance that you want to use in the format ServerName\Instance. You can also use the default entry. |
|
Database Name |
Type the name of the database.
The database name must be a unique name. |
|
Database Authentication |
Select the authentication that you want to use by doing one of the following:
In SQL authentication, an unencrypted password is sent to SQL Server. We recommend that you use SQL authentication only if you force protocol encryption to SQL Server or encrypt network traffic by using IPsec. |
- Click OK.
- Start the Machine Translation Service. For more information, see “Starting or stopping a service” in Manage services on the server (SharePoint Server 2010).
To create a Machine Translation service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
New-SPTranslationServiceApplication -Name “<ServiceApplicationName>” -DatabaseName “<DatabaseName>” -DatabaseServer “<DatabaseServer>” -ApplicationPool “<ApplicationPoolName>” -Default
Where:
- <ServiceApplicationName> is name of the new Machine Translation Service application.
-
<DatabaseName> is the name of the database that will host the Machine Translation Service logs. To create a new database, provide a new name.
Important:
The database name must be a unique name.
- <DatabaseServer> is the name of the database server that will hold the work items for the Machine Translation Service.
-
<ApplicationPoolName> is the name of an existing application pool in which the new Machine Translation Service should run.
Important:
The account that is used by the application pool must also have Full Control permissions to the User Profile service application. If you create a new application pool and a new account, make sure that you add the account to the list of accounts that can use the User Profile service application, and grant it Full Control permissions. For more information, see Restrict or enable access to a service application (SharePoint Server 2010).
Example
New-SPTranslationServiceApplication -Name “Machine Translation Service Application” -DatabaseName “MachineTranslationDB” -DatabaseServer “ContosoDBServer” -ApplicationPool “ContosoAppPool” -Default
- Start the Machine Translation Service. For more information, see “Starting or stopping a service” in Manage services on the server (SharePoint Server 2010).
For more information, see New-SPTranslationServiceApplication.
-
Configure the Machine Translation Service
You can configure the Machine Translation Service by using either Central Administration or Windows PowerShell.
Caution:
Changing the default settings for the Machine Translation Service can potentially affect server performance. For example, increasing item size limits can result in the translation job taking longer to run, and increasing the number of processes will consume more resources on the server. Be sure to carefully consider any possible server effects before you change these settings.
To configure the Machine Translation Service by using Central Administration
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group in SharePoint Server 2013.
- On the Central Administration home page, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click the link that corresponds to the name of the Machine Translation service application.
- On the Machine Translation Service page, in the Enabled File Extensions section, clear the check box for any file name extensions that you want to disable. By default, all file name extensions are enabled.
- In the Item Size Limits section, do the following:
- In the Maximum file size for binary files in KB. Microsoft Word documents are binary files box, type the maximum file size (100-524288), in KB, for binary files. The default is 51200. Files that exceed this limit will not be translated.
- In the Maximum file size for text files in KB. Plain-text, HTML, and XLIFF documents are text files box, type the maximum file size (100-15360), in KB, for text files. The default is 5120. Files that exceed this limit will not be translated.
- In the Maximum character count for Microsoft Word documents box, type the maximum character count (10000-10000000) for Word documents. The default is 500000.
- In the Online Translation Connection section, do one of the following:
- Click Use default internet settings. This is the default.
-
Click Use the proxy specified, and type a web proxy server and port number.
Note:
If you change this setting, you must stop and restart the Machine Translation Service after you configure it.
- In the Translation Processes section, type the number of translation processes (1-5). The default is 1.
Note:
If you change this setting, you must stop and restart the Machine Translation Service after you configure it.
- In the Translation Throughput section, do the following:
- In the Frequency with which to start translations (minutes) box, type the frequency with which groups of translations are started, in minutes (1-59). The default is 15.
- In the Number of translations to start (per translation process) box, type the number of translations (1-1000) per process. This number represents the number of translations started per process every time translations are started. The default is 200.
- In the Maximum Translation Attempts section, type the maximum number of times (1-10) a translation is tried before its status is set to Failed. The default is 2.
- In the Maximum Synchronous Translation Requests section, type the maximum number of synchronous translation requests (0-300). The default is 10.
Note:
You can also set this value to 0 so that no synchronous jobs are accepted.
- In the Translation Quota section, do the following:
-
In the Maximum number of items which can be queued in a 24-hour period section, do one of the following:
- Click No limit. This is the default.
- Click Limit per 24 hours, and then type the maximum number of items (100-1000000) that can be queued in a 24-hour period.
- Click No limit. This is the default.
-
In the Maximum number of items which can be queued in a 24-hour period per site subscription section, do one of the following:
- Click No limit. This is the default.
- Click Limit per 24 hours, and then type the maximum number of items (100-1000000) that can be queued in a 24-hour period per site subscription.
- Click No limit. This is the default.
Note:
This setting applies only if you will be providing hosting services for other sites, and the sites using it have site subscriptions.
- In the Completed Job Expiration Time section, do one of the following:
- Click Days, and then type the number of days (1-1000) completed jobs are kept in the job history log. The default is 7.
- Click No expiration.
- In the Recycled Threshold section, type the number of documents (1-1000) to be converted before the conversion process is restarted. The default is 100.
Note:
If you change this setting, you must stop and restart the Machine Translation Service after you configure it.
- In the Office 97-2003 Document Scanning section, specify whether to disable security scanning for Office 97-2003 documents. Only enable this setting if you trust the documents that will be converted. The default is No.
- Click OK.
- If you changed any settings that require you to restart the Machine Translation Service, restart the service now. For more information, see “Starting or stopping a service” in Manage services on the server (SharePoint Server 2010).
To configure the Machine Translation Service by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Set-SPTranslationServiceApplication -Identity “<ServiceApplicationName>” -EnableAllFileExtensions -UseDefaultlnternetSettings -TimerJobFrequency <TimerJobFrequency> -MaximumTranslationAttempts <MaximumTranslationAttempts> -JobExpirationDays <JobExpirationDays> -MaximumSyncTranslationRequests <MaximumSyncTranslationRequests> -RecycleProcessThreshold <RecycleProcessThreshold> -DisableBinaryFileScan <DisableBinaryFileScan>
Where:
- <ServiceApplicationName> is name of the Machine Translation service application.
- <TimerJobFrequency> is the frequency, in minutes (1-59), with which groups of translations are started.
- <MaximumTranslationAttempts> is the maximum number of times (1-10) a translation is tried before its status is set to Failed.
- <JobExpirationDays> is the number of days (1-1000) completed jobs are kept in the job history log.
- <MaximumSyncTranslationRequests> is the maximum number of synchronous translation requests (0-300).
- <RecycleProcessThreshold> is the number of documents (1-1000) to be converted before the conversion process is restarted.
-
<DisableBinaryFileScan> is either 0 (false) or 1 (true).
Example
Set-SPTranslationServiceApplication -Identity “Machine Translation Service Application” -EnableAllFileExtensions -UseDefaultlnternetSettings -TimerJobFrequency 30 -MaximumTranslationAttempts 3 -JobExpirationDays 14 -MaximumSyncTranslationRequests 20 -RecycleProcessThreshold 300 -DisableBinaryFileScan 1
Note:
Changes to any of the following parameters will require that you restart the Machine Translation Service: KeepAliveTimeout, MaximumTranslationTime, TotalActiveProcesses, RecycleProcessThreshold, WebProxyAddress, MachineTranslationAddress, UseDefaultInternetSettings.
- If you changed any settings that require you to restart the Machine Translation Service, restart the service now. For more information, see “Starting or stopping a service” in Manage services on the server (SharePoint Server 2010).
For more information, see Set-SPTranslationServiceApplication.
The Microsoft Translator Hub is an extension of Microsoft Translator, and allows you to build automatic language translation systems that integrate with your website. After you build a custom system, the Test System page on the Projects tab in the Microsoft Translator Hub displays a category ID. You can configure the Machine Translation Service to use the custom translation system by passing the category ID in the MachineTranslationCategory parameter. For more information about the Microsoft Translator Hub, see http://hub.microsofttranslator.com.
-
Additional steps
If the account that is used by the application pool that was assigned to the Machine Translation service application differs from the one used by the User Profile service application, you must add it to the list of accounts that can use the User Profile service application, and grant it Full Control permissions. For more information, see Restrict or enable access to a service application (SharePoint Server 2010).
-
Configure Request Manager in SharePoint Server 2013
Published: October 2, 2012
Summary: Learn how Request Manager in SharePoint Server 2013 can route and throttle incoming requests to help improve performance and availability.
Applies to: SharePoint Server 2013
Request Manager is functionality in SharePoint Server 2013 that enables administrators to manage incoming requests and determine how SharePoint Server 2013 routes these requests.
In this article:
-
Overview
Request Manager uses configured rules to perform the following tasks when it encounters requests:
- Deny potentially harmful requests from entering a SharePoint farm.
- Route good requests to an available server.
- Manually optimize performance.
Information that administrators or an automated process provide to Request Manager determine the effectiveness of routed requests.
To learn about how to use performance data to plan and manage the capacity of a SharePoint Server 2013 environment, see Capacity management and sizing overview for SharePoint Server 2013
- Deny potentially harmful requests from entering a SharePoint farm.
-
Scenarios
The following table describes possible scenarios and resolutions that Request Manager can address.
Area
Scenario
Resolution
Reliability and performance
Routing new requests to web front end with low performance can increase latency and cause timeouts.
Request Manager can route to front-end web servers that have better performance, keeping low performance front-end web servers available.
Requests from users and bots have equal priority.
Prioritize requests by throttling requests from bots to instead serve requests from end-users).
Manageability, accountability, and capacity planning
SharePoint Server fails or generally responds slowly, but it‘s difficult to identify the cause of a failure or slowdown.
Request Manager can send all requests of a specific type, for example, Search, User Profiles, or Office Web Apps, to specific computers. When a computer is failing or slow, Request Manager can locate the problem.
All front-end web servers must be able to handle the requests because they could be sent to any front-end web server.
Request Manager can send multiple or single requests to front-end web servers that are designated to handle them.
Scaling limits
Hardware scaling limited by load balancer
Request Manager can perform application routing and scale out as needed so that a load balancer can quickly balance loads at the network level.
-
Setup and Deployment
Request Manager’s task is to decide two things: a SharePoint farm will accept a request, and if the answer is “yes”, to which front-end web server SharePoint Server will send it. The three major functional components of Request Manager are Request Routing, Request Throttling and Prioritizing, and Request Load Balancing. These components determine how to handle requests. Request Manager manages all requests on a per-web-application basis. Because Request Manager is part of the SharePoint Server 2013 Internet Information Services (IIS) module, it only affects requests that IIS hosts.
When a new request is received, Request Manager is the first code that runs in a SharePoint farm. Although Request Manager is installed during setup of SharePoint Server on a front-end web server, the Request Management service is not enabled. You can use the Start-SPServiceInstance and Stop-SPServiceInstance cmdlets to start and stop the Request Management service instance respectively or the Manage services on server page on the the SharePoint Central Administration website. You can use the RoutingEnabled or ThrottlingEnabled parameters of the Set-SPRequestManagementSettings Windows PowerShell cmdlet to change properties of Request Manager.
Note:
There is no user interface to configure properties of Request Manager. The Windows PowerShell cmdlet is the only way to perform this task.
Request Manager has two supported deployment modes: Dedicated and Integrated.
-
Dedicated mode
Figure 1 shows a dedicated mode deployment.
Figure 1: Dedicated mode
A set of front-end web servers is dedicated to managing requests exclusively. The front-end web servers that are dedicated to Request Manager are in their own farm that is located between the hardware load balancers (HLBs) and the SharePoint farm. The HLBs send all requests to the Request Manager front-end web servers. Request Manager that runs on these front-end web servers decides to which SharePoint front-end web servers it will send the requests and then routes the requests. Depending on the routing and throttling rules, Request Manager might ignore some requests without sending them to another server. The SharePoint front-end web servers do their normal tasks in processing requests and then send responses back through the front-end web servers that run Request Manager and to the clients.
Note that all farms are set up as SharePoint farms. All front-end web servers in Figure 1 are SharePoint front-end web servers, each of which can do the same work as any other. The difference between the farms is that the Request Manager front-end web servers have Request Manager enabled.
Dedicated mode is good for larger-scale deployments when physical computers are readily available. The ability to create a separate farm for Request manager provides two benefits: Request Manager and SharePoint processes do not compete for resources and you can scale out one without having to also scale out the other. This allows you to have more control over the performance of each role.
- Request Manager and SharePoint processes do not compete for resources.
-
You can scale out each farm separately, which provides more control over the performance of each farm.
-
Integrated mode
-
Figure 2 shows an integrated mode deployment.
Figure 2: Integrated mode
In an integrated mode deployment, all SharePoint front-end web servers run Request Manager. Hardware load balancers send requests to all front-end web servers. When a front-end web server receives a request, Request Manager decides how to handle it: .
- Allow it to be processed locally.
- Route it to a different front-end web server.
- Deny the request.
Integrated mode is good for small-scale deployments when many physical computers are not readily available. This mode lets Request Manager and the rest of SharePoint Server to run on all computers. This mode is common for on-premises deployments.
-
-
Configuration
Request Manager has two configurable parts: General settings and Decision information. General settings are parameters that make Request Manager ready to use, such as enabling or disabling Request Routing and Request Throttling and Prioritizing. Decision information is all of the information that is used during the routing and throttling processes, such as routing and throttling rules.
Note:
You configure Request Manager on a farm and functionality occurs at a web application level.
-
General settings
By default, request routing and request throttling and prioritizing are enabled. You use the Set-SPRequestManagementSettings cmdlet to change the properties of request routing, request throttling and prioritizing, and select a routing weight scheme.
The table describes the configuration situation and Windows PowerShell syntax to use.
-
Windows PowerShell examples to enable routing and throttling
Situation
Windows PowerShell syntax
Enable routing and throttling for all web applications
Get-SPWebApplication | Set-SPRequestManagementSettings –RoutingEnabled $true –ThrottlingEnabled $true
Enable routing with static weighting for all web applications
Get-SPWebApplication | Get-SPRequestManagementSettings | Set-SPRequestManagementSettings –RoutingEnabled $true –ThrottlingEnabled $false –RoutingWeightScheme Static
In some situations, multiple front-end web servers will be suitable destinations for a particular request. In this case, by default, SharePoint Server selects one server randomly and uniformly.
One routing weight scheme is static-weighted routing. In this scheme, static weights are associated with front-end web servers so that Request Manager always favors a higher static weight during the selection process. This scheme is useful to give added weight to more powerful front-end web servers and produce less strain on less powerful ones. Each front-end web server will have a static weight associated with it. The values of the weights are any integer value, where 1 is the default. A value less than 1 represents lower weight, and greater than 1 represents higher weight.
Another weighting scheme is health-weighted. In health-weighted routing, front-end web servers that have health scores closer to zero will be favored, and fewer requests will be sent to front-end web servers that have a higher health score values. The health weights run from 0 to 10, where 0 is the healthiest and therefore will get the most requests. By default, all front-end web servers are set to healthy, and therefore, will have equal weights. SharePoint’s health score based monitoring system assigns weight to server and send a health score value as a header in the response to a request. Request Manager uses same health score and stores it in local memory.
-
Decision information
Decision information applies to routing targets, routing rules, and throttling rules.
-
Routing targets
Request routing determines the routing targets that are available when a routing pool is selected for a request. The scope of routing targets is currently for front-end web servers only, but Request Manager‘s design does not exclude routing to application servers, too. A list of front-end web servers in a farm is automatically maintained by using the configuration database. An administrator who wants to change that list, typically in dedicated mode, has to use the appropriate routing cmdlets to get, add, set, and remove routing targets.
The following table describes the various routing target tasks and the associated Windows PowerShell syntax to use.
-
Windows PowerShell examples routing target tasks
Task
Windows PowerShell syntax
Return a list of routing targets for all available web applications.
Get-SPWebApplication | Get-SPRequestManagementSettings | Get-SPRoutingMachineInfo –Availability Available
Add a new routing target for a specified web application.
Note:
IIS log files will contain all HTTP requests. For additional information about IIS logging, see IIS Logging
$web=Get-SPWebApplication -Identity <URL of web application>
$rm=Get-SPRequestManagementSettings -Identity $web
Add-SPRoutingMachineInfo –RequestManagementSettings $rm -Name <MachineName> -Availability Available
Where
- <URL of web application> is the URL of the web application to which you’re adding a new routing target.
- <MachineName>is the name of the server that hosts the web application.
Edit an existing routing target‘s availability and static weight for a specified web application
$web=Get-SPWebApplication -Identity <URL of web application>
$rm=Get-SPRequestManagementSettings -Identity $web
$m=Get-SPRoutingMachineInfo -RequestManagementSettings $rm -Name <MachineName>
Set-SPRoutingMachineInfo -Identity $m -Availability Unavailable
Where
- <URL of web application> is the URL of the web application for which you’re editing an existing routing target‘s availability and static weight.
Remove a routing target from a specified web application
Note:
You cannot remove front-end web servers that are in the farm. Instead, you can use the Availability parameter of the Set-SPRoutingMachineInfo cmdlet to make them unavailable.
$web=Get-SPWebApplication -Identity <URL of web application>
$rm=Get-SPRequestManagementSettings -Identity $web
$m=Get-SPRoutingMachineInfo -RequestManagementSettings $rm -Name <MachineName>
Remove-SPRoutingMachineInfo -Identity $M
Where
- <URL of web application> is the URL of the web application from which you’re removing a routing target.
-
Routing and throttling rules
Request routing and request throttling and prioritizing are decision algorithms that use rules to prescribe many actions. The rules determine how Request Manager handles requests.
Rules are separated into two categories, routing rules and throttling rules, which are used in request routing and request throttling and prioritizing, respectively. Routing rules match criteria and route to a machine pool. Throttling rules match criteria and throttle based on known health score of a computer.
-
-
Request Routing
Request processing is all operations that occur sequentially from the time that Request Manager receives a new request to the time that Request Manager sends a response to the client.
Request processing is divided into the components:
- request routing
- incoming request handler
- request throttling and prioritizing
-
request load balancing
-
Incoming request handler
-
The role of the incoming request handler is to determine whether Request Manager should process a request. If request throttling and prioritizing is disabled and the Request Manager queue is empty, Request Manager directs the request to SharePoint Server that is running on the current front-end web server. If request throttling and prioritizing is enabled, request throttling and prioritizing determines whether the request should be allowed or denied on the current front-end web server.
The processes steps of the incoming request handler are as follows:
- request routing
- Request is determined if it should be throttled or routed
- For routed requests, load balance algorithm is run
- Request routed to load balancer endpoint
Request routing and Request throttling and prioritizing only run if it is enabled and is routed once per farm. Request load balancer only runs if a request has been determined as routable. The outgoing request handler only runs if the request has to be sent to a different front-end web server. The role of the outgoing request handler is to send the request to the selected front-end web server, wait for a response, and send the response back to the source.
-
Request routing
The role of request routing is to select a front-end web server to route a request. By using no routing rules that are defined, the routing scheme is as easy as randomly selecting an available front-end web server.
The algorithm of request routing is defined by two parts: request-rule matching and front-end web server selection.
-
Request rule matching
Every rule has one or more match criteria, which consist of three things: match property, match type, and match value.
The following table describes the different types of match properties and match types:
|
Match property |
Match type |
|
Hostname |
RegEx |
|
URL |
Equals |
|
Port number |
Starts with |
|
MIME Type |
Ends with |
For example, an administrator would use the following match criteria to match http://contoso requests: Match Property=URL; Match value= http://contoso; Match type=RegEx
-
Front-end web server selection
The front-end web server selection uses all routing rules, whether they match or do not match a given request. Rules that match have machine pools, a request sends load balanced to any machine in any matching rule‘s machine pool. If a request does not match any request, it sends load balanced to any available routing target.
-
Request routing and prioritizing
For routing requests that use the health-based monitoring system, the role of request routing and prioritizing is to reduce the routing pool to computers that have a good health score to process requests. If request routing is enabled, the routing pool is whichever front-end web server is selected. If request routing is disabled, the routing pool only contains the current front-end web server.
Request routing and prioritizing can be divided into two parts: request-rule matching and front-end web server filtering. Request-rule matching happens exactly like in request routing. Front-end web server filtering uses the health threshold parameter from the throttling rules in combination with front-end web server health data to determine whether the front-end web servers in the selected routing pool can process the given request.
The front-end web server filtering process follows these steps:
- The routing pool is either the current front-end web server or one or more front-end web servers that request routing selects.
- All matching rules are checked to find the smallest health threshold value.
- Remove front-end web servers in the routing pool that have health scores greater than or equal to the smallest health threshold value.
For example, request routing is disabled and the current front-end web server has a health score of 7 and a rule “Block OneNote“ without a health threshold (that is, health threshold = 0) is created.
The routing pool is the current front-end web server that has a health threshold equal to zero (0). So, the smallest threshold that the front-end web server can serve is zero. Because the current front-end web server has health score of 7, Request Manager denies and removes the request.
-
Request load balancing
The role of request load balancing is to select a single target to which to send the request. Request load balancing uses the routing weight schemes to select the target. All routing targets begin with a weight of 1. If static weighting is enabled, request load balancing uses the static weights set of each routing target to adjust the weights and the value can be valid integer number. If health weighting is enabled, request load balancing uses health information to add weight to healthier targets and remove weight from less healthy targets.
-
Monitoring and maintenance
Monitoring and logging are keys to managing requests from Request Manager.
- The rules that matched.
- The rules that did not match.
-
The final decision of the request.
Decisions might include useful information such as the following.
- Was the request denied?
- Which front-end web server was selected and from which routing pool>
- Did the request succeed or fail and why?
- How long did each part, routing, throttling, and waiting for front-end web server to respond, take?
- Was the request denied?
An administrator can use this information to adjust the routing and throttling rule sets to optimize the system and correct problems. To help you monitor and evaluate your farm’s performance, you can create a performance monitor log file and add the following SharePoint Foundation Request Manager Performance counters:
Counter name
Description
Connections Current
The total number of connections that are currently open by Request Manager.
Connections Reused / Sec
The number of connections per second that are reused when the same client connection makes another request without closing the connection.
Routed Requests / Sec
The number of routed requests per second. The instance determines the application pool and server for which this counter tracks.
Throttled Requests / Sec
The number of throttled requests per second.
Failed Requests / Sec
The number of failed requests per second.
Average Processing Time
The time to process the request that is, the time to evaluate all the rules and determine a routing target.
Last Ping Latency
The last ping latency (that is, Request Manager’s PING feature) and the instance determine which application pool and machine target.
Connection Endpoints Current
The total number of endpoints that are connected for all active connections.
Routed Requests Current
The number of unfinished routed requests. The instance determines which application pool and machine target.
Along with creating a performance monitor log file, the verbose logging level can be enabled by using the following Windows PowerShell syntax:
Set-SPLogLevel “Request Management“
–TraceSeverity Verbose
- The rules that matched.
-
Configure Business Connectivity Services solutions for SharePoint 2013
Published: July 16, 2012
Summary: Find links to procedures to help you install and configure SharePoint 2013 Business Connectivity Services (BCS) for the on-premises and cloud-only solutions, and other scenarios.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This article is your starting place for the procedures to install common Microsoft Business Connectivity Services scenarios for SharePoint 2013. The Business Connectivity Services solution that you deploy will most likely look different from the solutions presented here, but you can model your installation on these examples. Also, you can select the individual procedures from here to build your own procedural documents for your Business Connectivity Services solution scenario.
-
About Business Connectivity Services installation scenarios
Every Business Connectivity Services solution is unique because each business has unique data integration problems that it solves with Business Connectivity Services. The solutions can range from something simple and straightforward that a power user or IT professional (who has the appropriate permissions) can perform by themselves, to complex solutions that require developer, IT professional, and end-user solution development involvement. This guide presents the configuration procedures in common scenarios.
- On-premises All the Business Connectivity Services components are under your organizations control behind your firewall.
- On-premises All the Business Connectivity Services components are under your organizations control behind your firewall.
-
Prerequisites
Before you begin with any Business Connectivity Services scenario configuration, make sure that you have read Business Connectivity Services Overview (SharePoint 2013 Preview) and completed the steps in Plan a Business Connectivity Services solution (SharePoint 2013 Preview).
-
On-premises deployment
The procedures in Deploy a Business Connectivity Services on-premises solution in SharePoint 2013 show you how to deploy a solution that involves the following:
- A Business Connectivity Services infrastructure that is on your corporate network.
- Information workers who access the Business Connectivity Services solution are on your corporate network.
- External content that is surfaced in SharePoint as an external list.
- External content that is synchronized into Outlook for offline use.
- Accessing external data that is in SQL Server database on your corporate network.
- SharePoint Designer 2013 to create the external content type for the SQL Server data source.
- The Secure Store Service to manage mapping of user credentials to group credentials for accessing the external systems.
- A Business Connectivity Services infrastructure that is on your corporate network.
-
Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
Updated: October 16, 2012
Summary: How to install Business Connectivity Services (BCS) to access an on-premises SQL Server external data source, surface external data in SharePoint lists, and take external data offline in Outlook.
Applies to: SharePoint Server 2013
The following scenario shows you how to create a no-code business solution in Microsoft Business Connectivity Services (BCS) by using the SQL Server AdventureWorks sample database. You learn how to:
- Configure the accounts, and groups that you need to securely access the external data source.
- Configure the permission on the external data source, the external content type and the external lists.
- Create and configure an external content type.
- Create an external list that makes the external data available to users.
- Connect the external list to Outlook to make the external data available when the user is offline.
This article includes the overview, roadmap, and any conceptual information that is needed.
- Configure the accounts, and groups that you need to securely access the external data source.
-
What these procedures help you deploy
Business Connectivity Services is a centralized infrastructure in SharePoint 2013 and Office 2013 that enables you to integrate data that is not in SharePoint products and Office 2013 into SharePoint products and Office 2013. BCS implementations take many different forms, including the on-premises form. These procedures show you how to install and configure BCS to integrate data from an on-premises SQL data source into a SharePoint products external list and into Outlook. For the purposes of building out this scenario, we use the AdventureWorks sample SQL database. The solution looks as shown in the following figure.
- A user goes to an external list on a SharePoint site. The external list creates a request for data by using the user‘s Windows credentials.
- The request is sent to the BDC runtime in the SharePoint farm.
- The BDC runtime accesses the external content type for the list (in the BDC Metadata Store) to see how to access the external system and which operations can be performed. By using either the user‘s credentials or the credentials from the Secure Store (as defined in the external content type), the BDC runtime passes the request to a connector that can handle the request, in this case the SQL connector.
- The SQL connector accesses the external data source and retrieves the data, and applies any formatting and filtering as specified in the external content type. The data is passed back through the request chain to the list where the user can interact with it.
- The user wants to take this data on a portable computer in Outlook so the user can use the Connect to Outlook feature on the external list to take the data offline.
- The Click Once installation runs and installs the required BDC model on the client. This lets the BDC Client-Side Runtime access the external data directly.
- Outlook then connects to the external data by using the configuration in the BDC model and synchronizes it into an Outlook SharePoint external list, formatted as a contacts list.
- The user can then interact with the contact data, and any changes that the user makes can be written back to the external data source either by an on-demand synch or by waiting six hours for the automated synchronization.
-
How to use these procedures and a roadmap of the procedures
The steps to completely deploy this scenario are presented in smaller procedures. Some of the procedures are on TechNet, some are on Office.com, and some are on MSDN. Each procedure is numbered indicating its position in the overall sequence. At the beginning and end of each procedure, links direct you to the preceding and following steps. The following list contains links to all of the procedures, in proper order, for your reference. You must follow them in sequence to build out the scenario. You can also use these procedures individually to build out your own unique scenarios. When you are assembling individual procedures to build out your own scenarios, be sure to test the entire set of procedures, in order, in a lab setting before you attempt them in production.
- Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013
- Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013
- Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013
- Create the Business Data Connectivity service application in SharePoint 2013
- Set permissions on the BCS Metadata Store for a Business Connectivity Services on-premises solution in SharePoint 2013
- Configure the Secure Store Service for a Business Connectivity Services on-premises solution in SharePoint 2013
- Create an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013
- Configure permission on an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013
- Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013
- Manage user permissions on an external list for a Business Connectivity Services on-premises solution in SharePoint 2013
- Connect an external list to Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013
- Verify offline access and synchronization of external data in Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013
-
Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to prepare your environment to install Business Connectivity Services (BCS) in an on-premises configuration in SharePoint 2013.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
Before you start your installation of the on-premises Microsoft Business Connectivity Services (BCS) scenario, you must have these software and infrastructure requirements in place.
Important:
This is Step 1 in the Business Connectivity Services On-Premises scenario deployment procedures.
- For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013.
- For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013.
-
On-premises scenario prerequisites
- A fully functional SharePoint 2013 server farm with a Web Application and site collection
- A fully functioning instance of SQL Server 2008 with Service Pack 1 (SP1) and Cumulative Update 2 or SQL Server 2008 R2
- SharePoint Designer 2013
- Outlook 2013 client
- You have an account that has permissions to administer the Business Data Connectivity Service Application
- You have an account that has permissions to administer the Secure Store Service service application
- Download the AdventureWorks sample database from Codeplex downloads for SQL Server. This database must be installed and functioning on your SQL Server
- Create an Active Directory directory Service (AD DS) security group and add the users who will be using this BCS solution, for example create a group that is named AdventureWorksBCSUsers
The AdventureWorks sample database is developed and published by Microsoft. The AdventureWorks sample database is prepopulated with a large quantity of fictitious data from a fictitious company, AdventureWorks Cycles. We are using the AdventureWorks sample database here so we have a concrete example for illustrating the installation and configuration of the on-premises BCS scenario.
Tip:
If you have problems attaching the database file by using the Attach command, in the Attach databases dialog box, and remove the reference to the log file in the bottom pane before you click OK.
- A fully functional SharePoint 2013 server farm with a Web Application and site collection
-
Preparing the environment
-
How to download and install the AdventureWorks sample database
-
- From a browser, go to AdventureWorks sample database and download the AdventureWorks2008R2_Data.mdf file.
- Install the Adventure Works2008R2 sample database by following the procedures in the “Readme for AdventureWorks 2008 R2 Sample Database“ section of the SQL Server Samples Readme (en-US) page.
Important:
Link to Step 2Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to prepare the SQL Server logins for Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Microsoft Business Connectivity Services (BCS) requires an account that it can use to access the external data source. The account must have the necessary permissions on the external data source to perform all the operations that your BCS solution might require. For ease of configuration and ongoing management, you can map a group of SharePoint products users to a single shared account on the external data source.
In this procedure, you create a SQL Server login and then assign that login to a user account on the AdventureWorks sample database. You will use Secure Store Service services to map a group of SharePoint 2013 users to the single shared account in a later procedure.
Important:
This is Step 2 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 1Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Create a SQL Server login
-
- Link to Step 1Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Start SQL Server Management Studio.
- In the Object Explorer, expand the <database server name>, expand Security, and then expand Logins.
- Right-click Logins, and then click New Login
- In the Login Name box, enter SharePointGroupAccount.
- Select SQL Server authentication, and then enter and confirm a password.
- In the Default database box, select AdventureWorks2008R2, and then click OK.
-
Create a SQL Server user on the AdventureWorks database
- In the Object Explorer, expand Databases, expand AdventureWorks2008R2, expand Security, and then expand Users.
- Right-click Users, and then click New User.
- Under the Login Name, with the User name box pre-selected, in the first box, enter AdventureWorksUser
- In the second box, click Browse, in the Select Login dialog box, click Browse, select the SQL Server account, SharePointGroupAccount, and then click OK twice.
- Under Database Role membership, select db_owner.
- Click OK.
- Close SQL Server Management Studio.
Important:
Link to Step 3Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to start the Business Data Connectivity service for a SharePoint 2013 server farm.
The Business Data Connectivity Service must be running for you to create any BCS based business solution. Use this procedure to start or stop the Business Data Connectivity Service
Important:
This is Step 3 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 2Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Start the Business Data Connectivity service
-
- Link to Step 2Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services
- Open the SharePoint Central Administration website for the server farm that contains your BCS solution.
- On the Quick Launch, click System Settings.
- On the System Settings page, under Servers, click Manage services on server.
- Check the value in the Server field. If the server name shown there is not the server that you want running the Business Data Connectivity Service on, click on the down arrow, click Change Server and select the correct server.
- If necessary, next to Business Data Connectivity Service, under the Action column, click Start.
Note:
If you need to stop the Business Data Connectivity Service after starting it, next to Business Data Connectivity Service in the Action column click Stop.
Important:
Link to Step 4Create the Business Data Connectivity service application in SharePoint 2013 of the Business Connectivity Services On-Premises deployment procedures
-
Create the Business Data Connectivity service application in SharePoint 2013
Published: October 16, 2012
Summary: How to create the Business Data Connectivity service application for SharePoint 2013 for a Business Connectivity Services on-premises configuration.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
Microsoft Business Connectivity Services is a SharePoint 2013 service application. You must create it if it was not created during your farms initial configuration.
Important:
This is Step 4 in the Business Connectivity Services on-premises scenario deployment procedures.
- Link to Step 3Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Create a new Business Data Connectivity Services service application
-
- Link to Step 3Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Open the SharePoint Central Administration website for your farm with a Farm administrator account. This must be the farm in which you started the Business Data Connectivity Service in the Start the Business Data Connectivity service for a Business Connectivity Services on-premises solution in SharePoint 2013 procedure.
- On the Quick start, click, Application Management.
- On the Application Management page under Service Applications, click Manage service applications.
- If an instance of the Business Data Connectivity Service Application that you will use for this solution is already there, you can skip the rest of this procedure. If not, follow the rest of this procedure to create one.
- On the SERVICE APPLICATIONS tab, click New and click Business Data Connectivity Service.
-
Configure the setting in the Create New Business Data Connectivity Service Application configuration page as follows:
- In the Service Application Name box enter the name you want the service to appear as on the Manage Service Applications page. This BCS service application can be used by multiple BCS solutions.
- In the Database area, leave the prepopulated values for Database Server, Database Name, and Database authentication, which is Windows authentication (recommended) unless you have specific design needs to change them.
- If you have SQL Server database mirroring configured and you want to include the Business Data Connectivity Service database in mirroring, provide the name of the failover database server in the Failover Database Server box.
- If you have not already created a new application pool for your service applications, enter a name for a new application pool in the Application pool name box, for example, “SharePointServiceApps“. You can use this application pool for all your service applications. For more information on planning, creating and configuring service applications, see Manage service applications in SharePoint 2013.
- Select the account that you configured in the Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013 procedure as the SharePoint products application services account in the Configurable drop down.
- In the Service Application Name box enter the name you want the service to appear as on the Manage Service Applications page. This BCS service application can be used by multiple BCS solutions.
- Click OK to create the new Business Data Connectivity Service Application and click OK again.
- Select the row that the Business Data Connectivity Service Application is in, not the proxy row.
- Click Administrators in the Operations area and add any accounts that you want to be able to administer the Business Data Connectivity service application granting them full control. When these individuals open Central Administration they will only be able to administer the Business Data Connectivity service application.
Important:
Link to Step 5Set permissions on the BCS Metadata Store for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Set permissions on the BCS Metadata Store for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to configure permissions on the Business Connectivity Services (BCS) Metadata Store for SharePoint 2013 for an on-premises configuration.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
The BCS Metadata Store holds external content types, external systems and BDC model definitions for the BCS Service Application. In this procedure you configure administrative permissions on the Metadata Store and everything that it will contain.
Important:
This is Step 5 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 4Create the Business Data Connectivity service application in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Set permissions on the Business Connectivity Services Metadata Store
-
- Link to Step 4Create the Business Data Connectivity service application in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Open the SharePoint Central Administration website with either a Farm administrator account or an account that has been delegated permissions to administer the Business Data Connectivity Service Applications.
- On the Quick Launch, click Application Management.
- On the Application Management page, under Service Applications, click Manage service applications.
- In the list of services, select the row of the Business Data Connectivity Service Application that you created in Create the Business Data Connectivity service application in SharePoint 2013 and then click Manage and then Set Metadata Store Permissions.
- Enter the Farm Administrator account and any other delegate administrators if you have them and then click Add.
- For each account or group that you added that is an administrator of the Business Data Connectivity Service Application, select the Edit, Execute, Selectable In Clients, and Set Permissions checkboxes.
- Select the Propagate permissions to all BDC Models, External Systems and External Content Types in the BDC Metadata Store. Doing so will overwrite existing permissions checkbox. For more information on setting permissions on the BDC Metadata Store, see Overview of Business Connectivity Services security tasks in SharePoint 2013.
- Click OK.
Note:
Edit is a highly privileged permission that is required to create or modify external content types in the Business Data Connectivity metadata store. Execute permission is required to query the external content type.
Important:
Link to Step 6Configure the Secure Store Service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Configure the Secure Store Service for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: Link to a configuration of the Secure Store Services for a Business Connectivity Services (BCS) on-premises solution in SharePoint 2013.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
The Secure Store Service stores the credentials that Microsoft Business Connectivity Services uses to access the AdventureWorks external data source and performs credential mapping between your users accounts and the credentials used to access the external data source.
Important:
This is Step 6 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 5Set permissions on the BCS Metadata Store for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
- Link to Step 5Set permissions on the BCS Metadata Store for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Parameters for configuring the Secure Store Service for a Microsoft Business Connectivity Services on-premises configuration
In this procedure you perform all the steps in Configure the Secure Store Services in SharePoint 2013 Preview article. You must perform the steps in the Configure the Secure Store Services in SharePoint 2013 article with these parameters from start to finish.
-
Configure Secure Store Service for on-premises Business Connectivity Services
-
- Perform all the steps in Configure the Secure Store Services in SharePoint 2013 Preview with the following parameters.
- Open the SharePoint Central Administration website for the server farm that your Secure Store Service is in with an account that has Farm Administrator permissions.
-
In the Configure the Secure Store Services in SharePoint 2013 Preview article, perform all procedures in the Configure Secure Store section with these parameters
- For the Register Managed Account, User name type in the name of the service account that you created in the Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013 procedure.
- Do not select the Enable automatic password change box.
- For the Register Managed Account, User name type in the name of the service account that you created in the Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013 procedure.
- Perform the “To start the Secure Store Service“ procedure
-
Perform the “To create a Secure Store Service application” procedures using these parameters
- In the Service Application Name box enter the name you want the service to appear as on the Manage Service Applications page.
- In the Database area, leave the prepopulated values for Database Server, Database Name, and Database authentication, which is Windows authentication (recommended) unless you have specific design needs to change them.
- If you have SQL Server database mirroring configured and you want to include the Secure Store Service in mirroring, provide the name of the failover database server in the Failover Database Server box.
- For the Configurable dropdown, select the account that you registered as a managed account earlier in this procedure.
- In the Service Application Name box enter the name you want the service to appear as on the Manage Service Applications page.
-
Perform the steps in the Work with encryption keys section with these parameters:
- Don‘t perform the procedures in the “Refresh the encryption key“ sub-section
- Don‘t perform the procedures in the “Refresh the encryption key“ sub-section
-
Read the Store credentials in Secure Store section and perform the Create a target application procedure using these parameters.
- In the Target Application ID box type in a string for the target application; this is not the display name. For example type in AWTargetAppID.
- In the Display Name box, enter the display name you want, for example Adventure Works Target Application ID.
- In the Target Application Type dropdown, select Group (which indicates the mapping of many credentials to one credential). In this case, the Target Application Page URL is not needed and automatically selects to None.
- On the Create New Secure Store Target Application page, under Field Name, change Windows User Name to SQL User Name, and Windows Password to SQL Password.
- Under Field Type change Windows User Name to User Name and change Windows Password to Password.
- In the Target Application Administrators add the accounts that you want to be administrators of the Target Application. Note that the Farm Administrator has access by default.
- In the Members box, add the names of the users whom you want to allow access to the external data source. For this example use the AdventureWorksBCSUsers security group you created in Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013.
- In the Target Application ID box type in a string for the target application; this is not the display name. For example type in AWTargetAppID.
-
Perform the steps in the Set credentials for a target application procedure using these parameters:
- In the SQL User Name box, type AdventureWorksUser which is the name SQL Server account you created in Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013.
- In the SQL Password, and Confirm SQL Password boxes type the password for that account, which is actually the password for the SharePointGroupAccount account that you created in Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013.
- In the SQL User Name box, type AdventureWorksUser which is the name SQL Server account you created in Create database logins for a Business Connectivity Services on-premises solution in SharePoint 2013.
Important:
Link to Step 7Create an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Create an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to create and configure an external content type for the Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to:
These procedures walk you through building an external content type for Business Connectivity Services using SharePoint Designer 2013 without writing any code. You will learn how to discover a SQL Server database, connect to the database table, and then return the required data. You will create an external content type named Customers that is based on the Customer view in the AdventureWorks sample database. This article uses the procedures in How to: Create external content types for SQL Server in SharePoint 2013 Preview. You must open that article and perform the steps there using the parameters given in the matching sections of this article.
Note:
The sections in this article match the sections in the How to: Create external content types for SQL Server in Sharepoint 2013.
Important:
This is Step 7 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 6Configure the Secure Store Service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
- Link to Step 6Configure the Secure Store Service for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Create and configure an external content type with SharePoint Designer 2013
-
Define general information
-
- Open How to: Create external content types for SQL Server in SharePoint 2013 Preview
- Create a new external content type named AWcustomers with a display name of AdventureWorks Customers.
-
Define general and Office behaviors
- Set the Office Item Type to Contact. The Office Item Type determines the Outlook behavior you want to attach to the external content type. In this case, this AWCustomer external content type behaves like a native Contact Item in Outlook.
- In the Offline Sync for External List checkbox, make sure Enabled is selected, which is the default.
Note:
If you disable this option, then the SharePoint Connect to Outlook ribbon command is not available for an external list.
-
Create a connection to the external data
- Add a connection using SQL Server as the External Data Source Type.
- In the Set the Database Server box, enter <The name of the database server> and in the Set the Database Name box, enter AdventureWorks2008R2. Optionally, in the Name box, enter AdventureWorks Sample Database.
- Select Connect with Impersonated Custom Identity.
- In the Secure Store Application ID box, enter AWTargetAppID.
Warning:
If you are prompted to enter a user name and password for AWTargetAppID it may be because when you created the SharePointGroupAccount SQL login, you did not uncheck the User must change password at next login option. To fix this, you must change the password via SQL query ALTER LOGIN <LoginName> WITH PASSWORD = ‘<originalpassword>‘
-
Select a table, view, or routine and Define Operation
- In the AdventureWorks Sample Database select the vIndividualCustomer view and right click Create All Operations.
Note:
Create All Operations is a convenient way to define all basic methods of operations (Create, Read, Read List, Update, and Delete).
Tip:
Always read carefully the messages in the Errors and Warnings pane. They provide useful information to confirm your actions or troubleshoot any issues.
-
Add columns
- In the Parameters Configuration dialog box, by default all columns are selected. To remove unnecessary columns, clear the checkboxes next to the following columns: Suffix and Demographics.
- For the BusinessEntityID select the Map to Identifier value.
Note:
Uncheck the Required box to prevent it from being updated but select the Read Only checkbox, which is needed to retrieve items so you can update other fields.
-
Map Outlook fields and set up the external item picker control
- For the FirstName, LastName, EmailAddress, and PhoneNumber fields, do the following:
- Click and highlight the field.
- Under properties, in the Office property dropdown, select the appropriate matching field: FirstName to First Name (FirstName), LastName to Last Name (LastName), and PhoneNumber to Primary Telephone Phone Number (PrimaryTelephonePhoneNumber), EmailAddress to EmailAddress1 (Email1Address).
Note:
Unmapped fields, depending on the number, are displayed as extended properties. For two to five fields they are listed as Adjoining meaning that they are appended to the form region at the bottom of an Outlook form’s default page. For six or more fields they are listed as Separate and are added as a new page to an Outlook.
- For the following fields, BusinessEntityID, FirstName, LastName, and EmailAddress click and highlight the field, and then under Properties, click Show in Picker.
-
Define filters
- Create a Comparison filter named ByRegion, use CountryRegionName for the value.
- Under Properties, next to Default Value, enter Canada.
- Create Limit filter named AWLimit, use BusinessEntityID for the Filter Field
- Set the default value to 200
Tip:
Click the Errors and Warnings pane and make sure there are no more errors or warnings.
-
Set the Title field for an external list and complete the external content type
- Set BusinessEntityID as the Title and save the external content type.
Important:
Link to Step 8Configure permission on an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Configure permission on an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: Configure permission on an external content type for a Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
To configure user access and permissions to the external content type:
Important:
This is Step 8 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 7Create an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Set up permissions to the external content type
-
- Link to Step 7Create an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Open the Central Administration page for your site.
- On the Quick Launch, click Application Management.
- On the Application Management page, under Service Applications, click Manage service applications.
- In the list of services, click your Business Data Connectivity (BDC) Service.
- Click AWCustomers.
- On the ribbon, click Set Object Permissions.
- Enter the user accounts to which you want to grant permissions, and then click Add. For this example, you would add the security group that was created in Prerequisites for deploying a Business Connectivity Services on-premises solution in SharePoint 2013AdventureWorksBCSUsers.
- Select the user accounts that you just added, and then select Execute check boxe.
- Select the Propagate permissions to all BDC Models, External Systems and External Content Types in the BDC Metadata Store check box to overwrite existing permissions.
- Click OK.
The external content type is now available for use in SharePoint and Office products to the appropriate users.
Important:
Link to Step 9Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: Create an external list, set its permissions and configure a view for a Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to:
An external list is a key building block for SharePoint solutions based on external data. You can read and write external data with the familiar experience of using a SharePoint list. An external list looks and behaves a lot like a native list, but there are some differences in behavior. For example, you can create views and use calculated values with formulas, but not attach files or track versions. For this exercise, you create the external list in the browser because that is a common approach. This article uses the procedures in Create an external list on Office.com. You must open that article and perform the steps there using the parameters given in the matching sections of this article.
Important:
This is Step 9 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 8Configure permission on an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Create an external list
-
- Link to Step 8Configure permission on an external content type for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Open Create an external list
- Create an external list named AdventureWorksCustomers using the AWCustomers external content type.
-
Create a view of an external list
- Create a view for the external list AdventureWorksCustomers. For this example use ByRegionData Source Filter.
- Make it the default view, and select your own Sort, Filter, and Limit values.
Important:
Link to Step 10Manage user permissions on an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Manage user permissions on an external list for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: How to configure permissions on external lists for a Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to:
Once you or an appropriate user has created the external list, it‘s important to make sure that you set appropriate permissions for other users. If the subsite that contains the external list inherits permissions from its parent site, then you may inadvertently give permission to inappropriate users. In this example, permissions are given to the AdventureWorksBCSUsers group.
Important:
This is Step 10 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 9Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Manage user permissions to the external list
-
- Link to Step 9Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- On the List tab, in the Settings group, click List Settings.
- Under Permissions and Management, click Permissions for this list…
- Apply permissions to the list as you have planned them.
The following table summarizes the default external list permissions for SharePoint user groups:
|
Name |
Permission levels |
|
Excel Services Viewers |
View Only |
|
<Site Name> Members |
Edit |
|
<Site Name> Owners |
Full Control |
|
<Site Name> Visitors |
Read |
Important:
Link to Step 11Connect an external list to Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Connect an external list to Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013
Published: October 16, 2012
Summary: Create a connection between an external list and Outlook in a Business Connectivity Services (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to:
The external list contains customer data mapped to Outlook contacts for which you enabled Offline Sync for External List, so you can connect the list with Outlook 2013. Once connected, you can view, edit, and print the data using the familiar Outlook user interface. This article mirrors the procedures in Connect an external list to Outlook on Office.com. Refer to that article for more information on connecting an external list to Outlook.
Important:
This is Step 11 in the Business Connectivity Services On-Premises scenario deployment procedures.
- Link to Step 10Manage user permissions on an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013.
-
Synchronize the external list with Outlook
-
- Link to Step 10Manage user permissions on an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- Open the SharePoint 2013 site that contains the external list. In the ribbon, on the List tab, in the Connect & Export group, click Connect to Outlook.
- In the Microsoft Office Customization Installer dialog box, click Install.The installation should take a minute or two.
- Once the installation is complete, click Close.
-
Link to
Step 12Verify offline access and synchronization of external data in Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
Verify offline access and synchronization of external data in Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013
Summary: How to work offline with external data in Outlook for a Business Data Connectivity (BCS) on-premises scenario deployment in SharePoint 2013.
Applies to:
Important:
This is Step 12 in the On-Premises scenario deployment procedures. This is the last step in the installation procedures for this scenario
- Link to Step 11Connect an external list to Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
-
For a list of all the procedures in order, see Deploy a Business Connectivity Services on-premises solution in SharePoint 2013
-
Update customer data offline and refresh it online
-
- Link to Step 11Connect an external list to Outlook for a Business Connectivity Services on-premises solution in SharePoint 2013Create an external list for a Business Connectivity Services on-premises solution in SharePoint 2013 of the Business Connectivity Services On-Premises scenario deployment procedures.
- To take Outlook 2013 offline, click Send/Receive, and in the Preferences group, click Work Offline.
- Make a change or two to one of the AdventureWorks customers.
- To bring Outlook 2013 back online, click Send/Receive, and in the Preferences group, click Work Online.
- To synchronize the data, on the navigation pane, right-click the <Team Site Name> AWCustomers external list and then click Sync now
-
Configure eDiscovery in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn the steps to set up and configure eDiscovery in SharePoint Server 2013 and Exchange Server 2013.
Applies to: SharePoint Server 2013
This article identifies the steps that are required to configure eDiscovery in SharePoint Server 2013. When you complete the steps that are listed in this article, users will be able to create and work with eDiscovery cases.
Before you configure eDiscovery, you should understand the concepts that are presented in the article Overview of eDiscovery and In-Place Holds (SharePoint 2013 Preview), and you should have completed the planning process that is described in the article Plan for eDiscovery.
The tasks that you must perform to configure eDiscovery are the following:
- Configure communication between SharePoint Server 2013 and Exchange Server 2013.
- Configure Search to crawl all discoverable content.
- Grant permissions.
- Create an eDiscovery Center.
- Configure communication between SharePoint Server 2013 and Exchange Server 2013.
-
Configure communication between SharePoint Server 2013 and Exchange Server 2013
If you will use a SharePoint eDiscovery Center to discover content in Exchange Server, you must configure SharePoint Server 2013 and Exchange Server to interact.
Important:
To discover content in Exchange Server from a SharePoint eDiscovery Center, you must be running Exchange Server 2013.
Perform the following steps:
- Ensure that the Exchange Web Service managed API is installed on every front-end server that is running SharePoint Server 2013. For more information about the Exchange Web Service managed API, see Hardware and software requirements (SharePoint 2013 Preview).
- Configure a trust relationship between SharePoint Server 2013 and Exchange Server. For information about how to configure the trust relationship, see Configure server-to-server authentication in SharePoint 2013.
- If you want content from Lync Server 2013 to be discoverable, configure Lync Server 2013 to archive to Exchange Server 2013. For information about how to configure Lync Server 2013 archiving, see Microsoft Lync Server 2013 Archiving Deployment Guide.
- Perform the eDiscovery configuration steps for Exchange. For information about how to configure Exchange Server 2013 for eDiscovery, see Configure Exchange for SharePoint eDiscovery Center.
-
Configure Search to crawl all discoverable content
Content is only discoverable if it is crawled and indexed by the Search service application that is associated with the web application that the eDiscovery Center is in. You should have identified this Search service application when you planned for eDiscovery. To configure the Search service application to crawl the appropriate content, follow these steps:
- If content in Exchange Server 2013 must be discoverable, add Exchange Server 2013 as a result source. For information about how to configure a result source, see Configure result sources for search in SharePoint Server 2013.
- Ensure that all websites that contain discoverable content are being crawled. For information about how to configure a location to be crawled, see Add, edit, or delete a content source (SharePoint Server 2010).
- Ensure that all file shares that contain discoverable content are being crawled. For information about how to configure a location to be crawled, see Add, edit, or delete a content source (SharePoint Server 2010).
-
Grant permissions
The article Plan for eDiscovery recommends that you create a security group to contain all users of the eDiscovery Center. After you create the security group, grant the security group permissions to access all discoverable content.
Note:
The article Plan for eDiscovery explains the different ways of granting permissions to discoverable content. You should have chosen to grant permissions at the web application level or at the site collection level.
- If you will grant permissions at the web application level, create a user policy that gives the security group full read permissions for each web application that contains discoverable content. For information about how to create a policy for a web application, see Manage permission policies for a Web application (SharePoint Server 2010).
Note:
When you change permissions at the web application level, Search re-crawls all of the content in the web application.
- If you will grant permissions at the site collection level, make the security group a site collection administrator for each site collection that contains discoverable content. For information about how to add a site collection administrator, see Add or change a site collection administrator.
Important:
A site collection administrator must add the security group as an additional site collection administrator by using the Site Settings menu. You cannot use Central Administration to make a security group a site collection administrator
- Ensure that the security group has permissions to access all file shares and other websites that contain discoverable content.
- If you will use a SharePoint eDiscovery Center to discover content in Exchange Server, grant the security group permissions to access Exchange Server mailboxes. For information about how to grant permissions in Exchange, see Configure Exchange for SharePoint eDiscovery Center.
- Grant the security group permissions to view the crawl log. For information about how to grant permissions to access the crawl log, see Set-SPEnterpriseSearchCrawlLogReadPermission.
-
Create an eDiscovery center
An eDiscovery Center is a site collection from which users can create and manage eDiscovery cases. To create an eDiscovery Center, follow the procedure in the article Create a site collection (SharePoint 2013 Preview), and choose the eDiscovery Center site collection type from the Enterprise tab. Be aware that an eDiscovery Center must be in a web application that supports claims authentication.
-
Configure site mailboxes in SharePoint Server 2013
Published: July 31, 2012
Summary: Configure Exchange Server 2013 and SharePoint Server 2013 for team email by using the SharePoint Server 2013 Site Mailboxes feature.
Applies to: Exchange Server 2013 | SharePoint Server 2013
This article describes how to configure Site Mailboxes in SharePoint Server 2013 and Exchange Server 2013. Site Mailboxes feature provides SharePoint Server 2013 users with team email on a SharePoint site. Site Mailboxes also provides links to SharePoint document libraries in Outlook 2013, enabling users to share files and email messages with other members of a team that are working on a joint project.
-
Before you begin
Before you begin this operation, review the following information about prerequisites:
- Site Mailboxes requires Exchange Server 2013.
- Any previous version of Exchange Web Services (EWS) will need to be uninstalled from the SharePoint servers.
Note:
You may need to determine if a previous version of EWS is installed. If so, please run the Check-SiteMailboxConfig script referenced below. Your version should be 15.0.516.25 or above.
- Site Mailboxes feature requires that user profile synchronization be configured in the farm. For information about configuring user profile synchronization, see Plan user profiles and identities (SharePoint Server 2013 Preview), and Manage user profile synchronization in SharePoint 2013 Server Preview.
- Site Mailboxes feature requires that the app management service application be configured in the farm. For information about configuring the app management service application, see New-SPAppManagementServiceApplication.
- Secure Sockets Layer (SSL) configured for the Default Zone is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication. This is such a scenario. As a prerequisite for configuring Site Mailboxes, the computer that is running SharePoint Server must have SSL configured. For more information, see Create claims-based web applications in SharePoint 2013 and follow the steps for creating an SSL site collection and server certificate.
Note:
You may need to import the Exchange Server SSL certificate from Exchange 2013 to SharePoint 2013, and from SharePoint 2013 to Exchange 2013. This is only necessary if the certificate is not trusted for the API endpoints (such as a Self-SSL Certificate in a lab environment).
To import an untrusted SSL certificate to a new server:
- Open Internet Explorer and navigate to Outlook Web App Preview (if on SharePoint) or the SSL SharePoint site (if on Exchange): https://<ExServerName>/owa or https://<SP_FQDN>.
- Accept to trust the certificate by clicking Continue to website.
- Click Certificate Error info in Internet Explorer next to the Address bar, and then click View Certificates.
- Select Install Certificate and then select Place all certificates in the following store.
- Select the checkbox to show physical stores.
- Install the certificate to Trusted Root Certification Authorities > Local Computer.
- In order to perform these procedures, you must be a member of the SharePoint and Exchange Server administrator groups and have an operational Exchange Server with end-user mailboxes.
- A SharePoint backup solution will not incorporate Exchange site mailboxes. An Exchange administrator will need to ensure timely backups of site mailboxes are taking place.
- Users who access files in a SharePoint document library from a Site Mailbox must have the document library configured as a trusted site in their browser or a warning will appear that asks the user if she or he wants to trust the file.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Site Mailboxes requires Exchange Server 2013.
-
Configure SharePoint for Site Mailboxes in SharePoint Server 2013
The first step in configuring Site Mailboxes is to install the Exchange Server Web Services API on each web front-end (WFE) server in the SharePoint Server 2013 farm.
-
Install Exchange Web Services API on SharePoint Server
-
- Download EWSManagedAPI.msi from the Microsoft Download Center (http://go.microsoft.com/fwlink/p/?LinkId=258305) and save it to a folder on each WFE server.
- Open a command window as administrator and navigate to the folder where you saved EWSManagedAPI.msi.
-
Run the following command:
msiexec /i EwsManagedApi.msi addlocal=”ExchangeWebServicesApi_Feature,ExchangeWebServicesApi_Gac”
- Reset IIS from the command line by typing IISReset.
-
Establish OAuth Trust and Service Permissions on SharePoint Server 2013
The next step is to copy the following two scripts. The first should be saved as Set-SiteMailboxConfig.ps1 and the second should be saved as Check-SiteMailboxConfig.ps1.
Set-SiteMailboxConfig.ps1:
# .SYNOPSIS
#
# Set-SiteMailboxConfig helps configure Site Mailboxes for a SharePoint farm
#
# .DESCRIPTION
#
# Establishes trust with an Exchange Server, sets Site Mailbox settings and enables Site Mailboxes for a farm.
#
# .PARAMETER ExchangeSiteMailboxDomain
#
# The FQDN of the Exchange Organization where Site Mailboxes will be created
#
# .PARAMETER ExchangeAutodiscoverDomain
#
# [Optional] The FQDN of an Exchange Autodiscover Virtual Directory
#
# .PARAMETER WebApplicationUrl
#
# [Optional] The URL of a specific web application to configure. If not specified all Web Applications will be configured
#
# .PARAMETER Force
#
# [Optional] Indicate that the script should ignore any configuration issues and enable Site Mailboxes anyway
#Param
(
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[string]$ExchangeSiteMailboxDomain,
[Parameter(Mandatory=$false)]
[ValidateNotNullOrEmpty()]
[string]$ExchangeAutodiscoverDomain,
[Parameter(Mandatory=$false)]
[ValidateNotNullOrEmpty()]
[string]$WebApplicationUrl,
[Parameter(Mandatory=$false)]
[switch]$Force
)$script:currentDirectory = Split-Path $MyInvocation.MyCommand.Path
if($WebApplicationUrl -ne $NULL -and $WebApplicationUrl -ne “”)
{
$webapps = Get-SPWebApplication $WebApplicationUrl
}
else
{
$webapps = Get-SPWebApplication
}if($webapps -eq $NULL)
{
if($WebApplicationUrl -ne $NULL)
{
Write-Warning “No Web Application Found at $($WebApplicationUrl). Please create a web application and re-run Set-SiteMailboxConfig”
}
else
{
Write-Warning “No Web Applications Found. Please create a web application and re-run Set-SiteMailboxConfig”
}return
}$rootWeb = $NULL
foreach($webapp in $webapps)
{
if($rootWeb -eq $NULL)
{
$rootWeb = Get-SPWeb $webApp.Url -EA SilentlyContinue
}
}if($rootWeb -eq $NULL)
{
Write-Warning “Unable to find a root site collection. Please create a root site collection on a web application and re-run Set-SiteMailboxConfig”
return
}$exchangeServer = $ExchangeAutodiscoverDomain
if($exchangeServer -eq $NULL -or $exchangeServer -eq “”)
{
$exchangeServer = “autodiscover.$($ExchangeSiteMailboxDomain)”
}Write-Host “Establishing Trust with Exchange Server: $($exchangeServer)”
$metadataEndpoint = “https://$($exchangeServer)/autodiscover/metadata/json/1”
$exchange = Get-SPTrustedSecurityTokenIssuer | Where-Object { $_.MetadataEndpoint -eq $metadataEndpoint }
if($exchange -eq $NULL)
{
$exchange = New-SPTrustedSecurityTokenIssuer -Name $exchangeServer -MetadataEndPoint $metadataEndpoint
}if($exchange -eq $NULL)
{
Write-Warning “Unable to establish trust with Exchange Server $($exchangeServer). Ensure that $($metadataEndpoint) is accessible.”if($ExchangeAutodiscoverDomain -eq $NULL -or $ExchangeAutodiscoverDomain -eq “”)
{
Write-Warning “If $($metadataEndpoint) does not exist you may specify an alternate FQDN using ExchangeAutodiscoverDomain.”
}
return
}Write-Host “Granting Permissions to Exchange Server: $($exchangeServer)”
$appPrincipal = Get-SPAppPrincipal -Site $rootWeb.Url -NameIdentifier $exchange.NameId
Set-SPAppPrincipalPermission -AppPrincipal $appPrincipal -Site $rootWeb -Scope SiteSubscription -Right FullControl -EnableAppOnlyPolicyWrite-Host
Write-HostWrite-Host “Verifying Site Mailbox Configuration”
$warnings = & $script:currentDirectory\Check-SiteMailboxConfig.ps1 -ReturnWarningStateif($warnings -and -not $Force)
{
Write-Warning “Pre-requisites not satisfied. Stopping Set-SiteMailboxConfig. Use -Force to override”
return
}
elseif($warnings)
{
Write-Warning “Pre-requisites not satisfied. -Force used to override”
}foreach($webapp in $webapps)
{
Write-Host “Configuring Web Application: $($webapp.Url)”
Write-Host “Setting Exchange Site Mailbox Domain to $($ExchangeSiteMailboxDomain)”
$webapp.Properties[“ExchangeTeamMailboxDomain”] = $ExchangeSiteMailboxDomainif($ExchangeAutodiscoverDomain -ne $NULL -and $ExchangeAutodiscoverDomain -ne “”)
{
Write-Host “Setting Exchange Autodiscover Domain to $($ExchangeAutodiscoverDomain)”
$webapp.Properties[“ExchangeAutodiscoverDomain”] = $ExchangeAutodiscoverDomain;
}$webapp.Update()
}$feature = Get-SPFeature CollaborationMailboxFarm -Farm -ErrorAction Ignore
if($feature -eq $NULL)
{
Write-Host “Enabling Site Mailboxes for Farm”
Enable-SPFeature CollaborationMailboxFarm
}
else
{
Write-Host “Site Mailboxes already enabled for Farm”
}
CheckSiteMailboxConfig.ps1:
Param
(
[Parameter(Mandatory=$false)]
[ValidateNotNullOrEmpty()]
[switch]$ReturnWarningState
)Add-PSSnapin Microsoft.SharePoint.Powershell
$anyWarnings = $false
Write-Host “Step 1: Checking for Exchange Web Services”
try
{
$assm = [System.Reflection.Assembly]::Load(“Microsoft.Exchange.WebServices, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35”)
if($assm.GlobalAssemblyCache)
{
Write-Host -Foreground Green “Found Exchange Web Services in Global Assembly Cache”
Write-Host “Exchange Web Services Version: $([System.Diagnostics.FileVersionInfo]::GetVersionInfo($assm.Location).FileVersion)”
}
else
{
Write-Warning “Unable to find Exchange Web Services in Global Assembly Cache”
$anyWarnings = $true
}
}
catch
{
Write-Warning “Unable to find Exchange Web Services in Global Assembly Cache”
$anyWarnings = $true
}Write-Host
Write-HostWrite-Host “Step 2: Checking for https web application”
$webapps = Get-SPWebApplication -EA SilentlyContinue
$rootWeb = $NULL
if($webapps -ne $NULL)
{
$sslWebAppExists = $false
foreach($webapp in $webapps)
{
if($rootWeb -eq $NULL)
{
$rootWeb = Get-SPWeb $webApp.Url -EA SilentlyContinue
}if(-not $webapp.Url.StartsWith(“https://”))
{
Write-Warning “Web Application at $($webapp.Url) does not use HTTPS. Site Mailboxes will not work on this Web Application.”
}
else
{
$sslWebAppExists = $true
Write-Host -Foreground Green “Found Web Application at $($webapp.Url) that uses HTTPS”
}
}if(-not $sslWebAppExists)
{
Write-Warning “At least one Web Application must be configured for HTTPS in the default zone.”
$anyWarnings = $true
}
}
else
{
Write-Warning “No Web Applications Found. Please create a web application and re-run Check-SiteMailboxConfig”
$anyWarnings = $true
if($ReturnWarningState)
{
return $anyWarnings
}
return;
}if($rootWeb -eq $NULL)
{
Write-Warning “Unable to find any Sites. Please create a root site collection on a web application and re-run Check-SiteMailboxConfig”
$anyWarnings = $true
if($ReturnWarningState)
{
return $anyWarnings
}
return;
}# Get App Permissions Management Objects
$appPrincipalManager = [Microsoft.SharePoint.SPAppPrincipalManager]::GetManager($rootWeb)
$appPrincipalPermissionsManager = New-Object -TypeName Microsoft.SharePoint.SPAppPrincipalPermissionsManager -ArgumentList $rootWebWrite-Host
Write-Host
Write-Host “Step 3: Checking for trusted Exchange Servers”$trustedIssuers = Get-SPTrustedSecurityTokenIssuer
$trustedIssuerHosts = @()if($trustedIssuers -ne $NULL)
{
$foundTrustedIssuer = $false
foreach($trustedIssuer in $trustedIssuers)
{
if($trustedIssuer.RegisteredIssuerName.StartsWith(“00000002-0000-0ff1-ce00-000000000000@”))
{
if($trustedIssuer.IsSelfIssuer)
{
$foundTrustedIssuer = $true$uri = New-Object -TypeName System.Uri -ArgumentList $trustedIssuer.MetadataEndPoint
Write-Host -Foreground Green “Found trusted Exchange Server at $($uri.Host)”
$appPrincipalName = [Microsoft.SharePoint.SPAppPrincipalName]::CreateFromNameIdentifier($trustedIssuer.RegisteredIssuerName)
$appPrincipal = $appPrincipalManager.LookupAppPrincipal([Microsoft.SharePoint.SPAppPrincipalIdentityProvider]::External, $appPrincipalName);if($appPrincipal -ne $NULL)
{
$isValidAppPrincipal = $true;if($appPrincipalPermissionsManager.GetAppPrincipalSiteSubscriptionContentPermission($appPrincipal) -eq [Microsoft.SharePoint.SPAppPrincipalPermissionKind]::FullControl)
{
Write-Host -Foreground Green “Exchange Server at $($uri.Host) has Full Control permissions”}
else
{
Write-Warning “Exchange Server at $($uri.Host) does not have Full Control permissions”
$isValidAppPrincipal = $false;
$anyWarnings = $true
}if($appPrincipalPermissionsManager.IsAppOnlyPolicyAllowed($appPrincipal))
{
Write-Host -Foreground Green “Exchange Server at $($uri.Host) has App Only Permissions”
}
else
{
Write-Warning “Exchange Server at $($uri.Host) does not have App Only Permissions”
$isValidAppPrincipal = $false;
$anyWarnings = $true
}if($isValidAppPrincipal)
{
$trustedIssuerHosts += $uri.Host
}}
else
{
Write-Warning “Unable to get App Principal for $($uri.Host). Unable to check permissions for this Exchange Server”
$anyWarnings = $true
}
}
else
{
Write-Warning “Found trusted Exchange Server at $($uri.Host) but it is not a Self Issuer”
$anyWarnings = $true
}
}
}if(-not $foundTrustedIssuer)
{
Write-Warning “Unable to find any trusted Exchange Servers”
$anyWarnings = $true
}
}
else
{
Write-Warning “Unable to find any trusted Exchange Servers”
$anyWarnings = $true
}Write-Host
Write-Host
Write-Host “Step 4: Report current Site Mailbox Configuration”if($webapps -ne $NULL)
{
foreach($webapp in $webapps)
{
Write-Host
Write-Host “Web Application Site Mailbox Configuration: $($webapp.Url)”
Write-Host “Exchange Site Mailbox Domain: $($webapp.Properties[“ExchangeTeamMailboxDomain”])”if($webapp.Properties[“ExchangeAutodiscoverDomain”] -ne $NULL)
{
Write-Host “Exchange Autodiscover Domain: $($webapp.Properties[“ExchangeAutodiscoverDomain”])”
}
}
}Write-Host
Write-Host “Trusted Exchange Services: $([String]::Join(“, “, $trustedIssuerHosts))”$feature = Get-SPFeature CollaborationMailboxFarm -Farm -ErrorAction Ignore
if($feature -eq $NULL)
{
Write-Host -ForegroundColor Red “Site Mailboxes are NOT enabled for Farm”
}
else
{
Write-Host -ForegroundColor Green “Site Mailboxes are enabled for Farm”
}if($ReturnWarningState)
{
return $anyWarnings
}
Save the two .ps1 files to the same folder on a SharePoint 2013 WFE server, as one script calls the other during execution. In a SharePoint PowerShell window (right-click and Run As Administrator to open), navigate to the folder containing the .ps1 files and run the Set-SiteMailboxConfig.ps1 script. This will allow users to retrieve and install the Exchange metadata, giving the Exchange service principal full control permissions to SharePoint site subscription, enable the site mailbox feature in the SharePoint environment and optionally set the Exchange site mailbox target domain, if DNS for the domain has not been configured for AutoDiscover. The Check-SiteMailboxConfig.ps1 is called as part of the Set-SiteMailboxConfig script, and will confirm the configuration has been successful (it can also be run separately).
The format should be as follows:
.\Set-SiteMailboxConfig.ps1 <Domain> <Exchange Server> [URL] [FQDN of the Exchange AutoDiscovery virtual directory]
Where <Domain> will equal the FQDN of the domain your Exchange is in, and <Exchange Server> is the Exchange you intend to connect to. These are required parameters.
Optional parameters are [URL], which would be a specific URL you may be configuring (typically used in an environment with SSL and non-SSL web applications), while [FQDN of the Exchange AutoDiscovery virtual directory] may need to be configured if DNS AutoDiscovery is not enabled or properly configured.
Example: .\Set-SiteMailboxConfig.ps1 tailspintoys.com exchange1.tailspintoys.com https://tailspintoys.com https://exchange1.tailspintoys.com/autodiscover/metadata/json/1If while running the script you encounter an error, please refer to the Troubleshooting section below for guidance.
-
Configure Exchange Server 2013 for Site Mailboxes
The final step is to establish OAuth trust, and service permissions, on the Exchange server.
-
Establish OAuth Trust and Service Permission on Exchange
-
- On your Exchange Server open the Exchange Windows PowerShell window as Administrator and change to the “C:\Program Files\Microsoft\Exchange Server\V15\Scripts” directory.
-
Run the following command:
.\Configure-EnterprisePartnerApplication.ps1 -ApplicationType Sharepoint -AuthMetadataUrl https://<SP_FQDN>/_layouts/15/metadata/json/1
Where <SP_FQDN> is the URL to the SharePoint SSL root site collection you wish to configure.
-
Troubleshooting
Please review the following if issues are encountered.
-
Table of Error Codes for Reference When Running Configuration Checklist Script
Error Code
Error
Notes
0
NoError
Review Prerequisites.
1
ExchangeClientNotAvailable
EWS client was not found on the SharePoint WFE. Run the Check script and ensure the entries are properly in the GAC; you may need to reinstall the EWS client.
2
UnsupportedVersion
EWS client version is incompatible with SharePoint. Run the Check script to ensure the version meets minimum requirements. Alternatively, the Exchange server may be 2010 or earlier.
3
InvalidUser
The TeamMailboxDomain parameter is not a valid FQDN or SMTP address.
4
UnauthorizedUser
The script received a 401 from the Exchange Server, review the Exchange setup steps.
5
ServerBusy
Exchange timed out during AutoDiscovery. It should be intermittent, please retry, but if it is persistent, follow-up with the Exchange Administrator.
6
URLNotAvailable
AutoDiscovery failed to return a URL for ECP/OWA, which means typically that the EWS client version is incompatible with SharePoint. It may also mean Site Mailboxes are not enabled on Exchange, which would require follow-up with the Exchange Administrator.
7
OAuthNotSupported
Unsuccessful in generating an OAuth token on behalf of SharePoint. This is typically caused by claims-based authentication being disabled on the SharePoint web application.
8
OAuthException
An error occurred during the OAuth handshake between SharePoint and Exchange. This is typically caused by server to server configuration issues, such as a realm value mismatch on either side, certificate issues for Exchange or SharePoint, etc. Review certificates and attempt to establish or reestablish trust.
9
InvalidAutodiscoverDomain
The AutoDiscover domain property is not set to a valid FQDN.
10
UnknownError
An unknown error condition has occurred. Run the Check script and confirm that a valid, trusted instance of SharePoint is available, review prerequisites, confirm AutoDiscover has been set-up properly with the Exchange Administrator.
101
OAuthNotSupportedOverHttp
If this error is thrown, your web application‘s default zone is not set to SSL, and AllowOauthoverHttp is also set to false. Run the Check script to ensure that any web application you intend to host site mailboxes are set with SSL in the default zone, as outlined in the prerequisites.
102
AssociatedOwnersGroupNull
One or both of the default Owners and Members groups for the site have been deleted. Each of these two default groups are required to exist on any site where users install site mailboxes. A site administrator should be able to direct a site owner to recreated these required groups.
103
ExchangeTeamMailboxDomainNotSet
The ExchangeTeamMailboxDomain property has not been set.
104
ExchangeAppPrincipalNotFound
No Exchange app principals were found to be trusted. Typically, this means the New-SPTrustedSecureTokenService step was missed. Run the Check script and ensure that the app principal URL(s) outputted are the correct one(s).
105
ExchangeAppPrincipalMissingPermissions
The Exchange app principal being connected to doesn‘t have the right permissions on the SharePoint farm. Run the Check script and ensure that the Exchange app principal has the required permissions on the farm.
-
-
Configure Exchange task synchronization in SharePoint Server 2013
Published: August 21, 2012
Summary: Configure Exchange Server 2013 and SharePoint Server 2013 for task synchronization by using the SharePoint Server 2013 Task Synchronization feature.
Applies to: SharePoint Server 2013 Enterprise
This article describes how to configure Task Synchronization in SharePoint Server 2013 and Exchange Server 2013. Task Synchronization allows users to synchronize SharePoint Server 2013 and Project Server tasks with Exchange Server and have them appear in Outlook 2013.
-
Before you begin
Before you begin this operation, review the following information about prerequisites:
- Task Synchronization requires that user profile synchronization be configured in the farm. For information about configuring user profile synchronization, see Plan user profiles and identities (SharePoint Server 2013 Preview), and Manage user profile synchronization in SharePoint 2013 Server Preview.
- Task Synchronization requires that the work management service application be configured in the farm. For information about creating the work management service application, see New-SPWorkManagementServiceApplication
- Task Synchronization requires Exchange Server 2013.
- Secure Sockets Layer (SSL) is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication. This is such a scenario. As a prerequisite for configuring Task Synchronization, the computer that is running SharePoint Server must have SSL configured. For more information, see Create claims-based web applications in SharePoint 2013 and follow the steps for creating an SSL site collection and server certificate.
Note:
You may need to import the SSL certificate from the SharePoint Server 2013 web application. This is only necessary if the certificate is not trusted for the API endpoints (such as a Self-SSL Certificate in a lab environment).
To import the untrusted SSL certificate from SharePoint Server 2013:
- Open Internet Explorer on the Exchange server and navigate to the SSL SharePoint site https://<SP_FQDN>, where <SP_FQDN> is the URL to the SSL site.
- Accept to trust the certificate by clicking Continue to website.
- Click Certificate Error info in Internet Explorer next to the Address bar, and then click View Certificates.
- Select Install Certificate and then select Place all certificates in the following store.
- Select the checkbox to show physical stores.
- Install the certificate to Trusted Root Certification Authorities > Local Computer.
- In order to perform these procedures, you must be a member of the SharePoint and Exchange Server administrator groups and have an operational Exchange Server with end-user mailboxes.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Task Synchronization requires that user profile synchronization be configured in the farm. For information about configuring user profile synchronization, see Plan user profiles and identities (SharePoint Server 2013 Preview), and Manage user profile synchronization in SharePoint 2013 Server Preview.
-
Configure SharePoint for Task Synchronization in SharePoint Server 2013
The first step in configuring Task Synchronization is to install the Exchange Server Web Services API on each web front-end server in the SharePoint Server 2013 farm.
-
Install Exchange Web Services API on SharePoint Server
-
- Download EWSManagedAPI.msi from the Microsoft Download Center (http://go.microsoft.com/fwlink/p/?LinkId=258305) and save it to a folder on the application server.
- Open a command window as administrator and navigate to the folder where you saved EWSManagedAPI.msi.
-
Run the following command:
msiexec /i EwsManagedApi.msi addlocal=”ExchangeWebServicesApi_Feature,ExchangeWebServicesApi_Gac”
- Reset IIS from the command line by typing IISReset.
-
Configure Exchange Server 2013 for Task Synchronization
The next step is to establish OAuth trust and service permission on Exchange Server.
-
Establish OAuth Trust and Service Permission on Exchange
-
- On the Exchange server, open Windows PowerShell and change to the “C:\Program Files\Microsoft\Exchange Server\V15\Scripts” directory.
-
Run the following script:
.\Configure-EnterprisePartnerApplication.ps1 -ApplicationType Sharepoint -AuthMetadataUrl https://<SP_FQDN>/_layouts/15/metadata/json/1
Where <SP_FQDN> is the URL to the root site collection.
-
Configure social computing features in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to configure social computing features in SharePoint 2013, including My Sites, Community Sites, and microblogging.
Applies to: SharePoint Server 2013
SharePoint Server 2013 implements features that make enterprise social computing and collaboration easier. Social networking tools, such as My Sites, and social content technologies, such as microblogs, are examples of social computing features. These features enable users to easily capture and share the knowledge and expertise that is needed to do their work. This sharing of information encourages collaboration, improves innovation, and targets relevant content to the people who have to see it. You can adapt content to each user while enabling administrators to set policies to protect privacy.
-
TechNet articles about configuring social computing features
The following articles about how to configure social computing features in SharePoint Server 2013 are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Learn how to set up My Sites in SharePoint Server 2013.
Learn how to set up Community Sites in SharePoint Server 2013.
Learn how to configure microblogging in SharePoint Server 2013.
Enable or disable personal and social features for users or groups in SharePoint Server 2013
Learn how to configure user permissions for personal and social features in SharePoint Server 2013.
-
Additional resources about configuring social computing features
The following resources about how to configure social computing features in SharePoint Server 2013 are available from other subject matter experts.
Content
Description
Visit the Resource Center to access videos, community sites, documentation, and more.
-
Configure My Sites in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to set up and configure My Sites in SharePoint Server 2013.
Applies to: SharePoint Server 2013
This article describes how to set up My Sites in SharePoint Server 2013. Like other tasks in SharePoint Server, there are multiple ways to complete a task. This article provides ordered tasks with prerequisites and procedures to help you set up My Sites in your enterprise.
Before you set up My Sites, ensure that you understand the concepts and terminology in My Sites overview (SharePoint Server 2010) and Plan for My Sites (SharePoint Server 2010).
We recommend that you perform all of the procedures in the order listed for best results, although not all of them are required.
In this article:
- Prerequisites
- Create a My Site host site collection
- Add a wildcard inclusion managed path to the web application
- Connect the web application to service applications
- Enable self-service site creation for the web application
- Configure My Site settings for the User Profile service application
- Enable the User Profile Service Application – Activity Feed Job
- Next steps
- Prerequisites
-
Prerequisites
Because My Sites have dependencies on other service applications and features in SharePoint Server 2013, ensure that you meet the prerequisites in this section before you perform the procedures in this task.
Note:
My Sites are hosted by a web application and rely on a User Profile service application. Both are described in this section. My Sites also requires a managed metadata service application. We recommend that you also have a Search service application to use with My Sites, but this is not required. Without the Search service application, some My Sites functionality is affected. For more information, see Related service applications in Plan for My Sites (SharePoint 2013 Preview).
Although you can use an existing web application, for optimal performance and security, we recommend that you create the My Site host site collection in a dedicated web application. For more information, see Create a Web application (SharePoint Server 2010).
Important:
If a My Site host site collection was created during initial deployment and configuration, we recommend that you do not use it because it was created in the default web application. Delete this site collection, and create a new web application that is dedicated to hosting My Sites. Then create a new My Site host site collection in the dedicated web application.
Ensure you have a User Profile service application that you want to use for My Sites. If you do not, follow the steps in Create, edit, or delete a User Profile service application (SharePoint 2013 Preview) to create one.
Important:
Although the Create New User Profile service application dialog box requests information in the My Site Host URL and Personal Site Location sections, for this task, remove any default values and leave those fields blank when you create the User Profile service application. Additionally, you can select any of the options in Site Naming Format. These settings will be configured separately later in this task.
Optionally, configure profile synchronization if you want to synchronize user and group profile information that is stored in the SharePoint Server 2013 profile database with profile information that is stored in a directory service or business system. For more information, see Plan for profile synchronization (SharePoint 2013 Preview).
-
Create a My Site host site collection
The My Site host site collection is a site collection that uses the Enterprise site template named My Site Host. This site collection must be created in the web application that you want to host My Sites. Generally, this site collection can be created at the root path of the web application, although it can be created as an explicit inclusion managed path deeper in the URL as long as there is a site collection created at the web application root. For more information about how to select the path for the My Site host collection, see My Sites architecture in Plan for My Sites (SharePoint 2013 Preview).
To create a My Site host site collection
- Verify that you have the following administrative credentials:
- To create a My Site host site collection, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website or a service application administrator for the services related to My Sites. If you are a service application administrator, you must also have permission to create site collections in the web application that you dedicate to host My Sites.
- In Central Administration, click Application Management, and then click Create site collections.
- On the Create Site Collection page, in the Web Application section, ensure that the selected web application is the web application that you want to host My Sites. If it is not, expand the list, and then click Change Web Application. In the Select Web Application dialog box, select a different web application.
- In the Title and Description section, type a title and description for the site collection.
- In the Web Site Address section, select the URL where you want this site collection created. Generally, you should use the default path (which is displayed as / in the user interface), which is the root of the web application. For more information about this path, see My Sites architecture in Plan for My Sites (SharePoint 2013 Preview).
- In the Template Selection section, in the Select experience version list, select 2013. Then, on the Enterprise tab, click My Site Host.
- In the Primary Site Collection Administrator section, and optionally in the Secondary Site Collection Administrator section, type an account in the format domain\username to specify an administrator for the site collection.
- Optionally, in the Quota Template section, select a quota template for the My Site host site collection. This quota template does not affect the individual site collections that users create for their My Sites. For more information, see Planning for storage requirements in Plan for My Sites (SharePoint 2013 Preview).
- Click OK. Copy this site collection URL for later reference.
-
Add a wildcard inclusion managed path to the web application
The wildcard inclusion managed path is the path under which separate site collections are created for a user’s My Site. Creation of the site collection occurs the first time that a user views the user‘s My Site. This functionality is available only when self-service site creation is also enabled. Enabling self-service site creation is discussed later in this article. For more information about managed paths, see Define managed paths (SharePoint Server 2010).
To add a wildcard inclusion managed path to the web application
- Verify that you have the following administrative credentials:
- To add managed paths, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website.
- In Central Administration, click Application Management, and then click Manage Web applications.
- On the Web Applications Management page, select the web application that you created to host My Sites.
- On the Web Applications tab, in the Manage group, click Managed Paths.
- In the Define Managed Paths dialog box, in the Add a New Path section, in the Path box, type the path that you want to append to the URL namespace, and then select Wildcard inclusion. For example, if your web application URL is http://mysites.contoso.com/ and you want users’ individual site collections created under a path named “personal”, type personal in the Path box. Separate My Sites site collections will be created for each user under http://mysites.contoso.com/personal/.
- Click Add Path, and then click OK.
- Copy this managed path for later reference.
-
Connect the web application to service applications
The web application that hosts My Sites must be connected to service applications in SharePoint Server 2013. The User Profile service application is required for My Sites. The managed metadata service application and Search service application are highly recommended. For more information, see My Sites architecture in Plan for My Sites (SharePoint 2013 Preview).
Additionally, if you have other SharePoint sites from which you want users to be able to access their My Site and About Me links from the upper-right corner menu, connect the web applications of those sites to the User Profile service application.
To connect the web application to service applications
- Verify that you have the following administrative credentials:
- To connect a web application to a service application, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website.
- In Central Administration, in the Application Management section, click Manage Web applications.
- On the Web Applications Management page, select the web application that you created to host My Sites.
- On the Web Applications tab, in the Manage group, click Service Connections.
- In the Configure Service Application Associations dialog box, in the Edit the following group of connections list, select default if the default group contains the service applications that you want to connect to the web application.
- If you choose [Custom], select any service applications to which you want to connect the web application, including the User Profile service application, the managed metadata service application, and the Search service application.
- Click OK.
-
Enable self-service site creation for the web application
Self-service site creation enables the automatic creation of a separate site collection for users when they first view their My Site.
To enable self-service site creation for the web application
- Verify that you have the following administrative credentials:
- To enable self-service site creation, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website.
- In Central Administration, in the Application Management section, click Manage Web applications.
- On the Web Applications page, select the web application that you created to host My Sites.
- On the Web Applications tab, in the Security group, click Self-Service Site Creation.
- In the Self-Service Site Creation Management dialog box, in Site Collections, select On. Optionally, in Quota template to apply, select a quota template.
-
In Start a Site, choose one of the following options:
- Prompt users to create a team site under so users can create team sites from their My Site to use site feeds.
- Be hidden from users if you do not want users to create team sites from their My Sites to use site feeds.
- Prompt users to create a team site under so users can create team sites from their My Site to use site feeds.
- Click OK to finish.
Perform these additional steps to configure permissions for users to create team sites from their My Sites to use site feeds.
- In the Policy group, click Permission Policy.
- On Manage Permission Policy Levels dialog box, click Add Permission Policy Level.
- Type a name for the permission policy.
- Under Permissions, in Site Permissions, select the Grant option for Create Subsites – Create subsites such as team sites, Meeting Workspace sites, and Document Workspace sites.
- Click Save.
- In the Policy group, click User Policy.
- On Policy for Web Application dialog box, click Add Users.
- On Add Users, in Zones select (All Zones), then click Next.
- In Choose Users, enter the user names of the users that you want to create team sites from their My Site to use site feeds. If all users can create team sites from their My Site to use site feeds, click the Browse icon. In Select People and Groups, click All Users, then click Everyone. Click Add, and then click OK.
- In the Choose Permissions section, select the name of the Permission Policy created previously.
- Click Finish, and then click OK.
-
Configure My Site settings for the User Profile service application
After you have a My Site host site collection and wildcard inclusion managed path configured for My Sites, you can update the My Sites settings in the User Profile service application. Most of these settings are configured during initial deployment and only change infrequently during maintenance operations afterward.
To configure My Site settings for the User Profile service application
- Verify that you have the following administrative credentials:
- To configure My Site settings for the User Profile service application, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website or a service application administrator for the User Profile service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- Click the User Profile service application that you connected to the web application hosting My Sites earlier in this task.
- On the Manage Profile Service page, in the My Site Settings section, click Setup My Sites.
- On the My Sites Settings page, in the Preferred Search Center section, specify settings for the search center to direct users to when they search for people or documents from their About Me profile page. If you do not have a search center set up yet, you can skip this step and complete it later. For more information, see Search service application in Plan for My Sites (SharePoint 2013 Preview).
- In the My Site Host section, type the URL of the My Site host site collection that you created earlier in this task.
- Optionally, in the My Site Host URL in Active Directory section, type the URL of the My Site host site collection that is returned to client and mobile phone applications that uses Exchange Auto Discovery. When a user is using a client or mobile phone application, credentials are passed in the form of an email address and password. Exchange Auto Discover then finds other required settings, such as SMTP server name, and sends this to the client or mobile phone application. Client and mobile phone applications use Exchange Auto Discovery to find a user’s SharePoint Server 2013My Site based on the My Site host URL stored in Active Directory Domain Services (AD DS).
- In the Personal Site Location section, type the wildcard inclusion managed path you configured earlier in this task. By default, personal is prepopulated in the box. However, if you chose a different path for your wildcard inclusion managed path, replace personal with your path.
- In the Site Naming Format section, select a naming format for the My Sites site collections that will be created when users view their My Sites for the first time. For more information about these formats, see My Sites architecture in Plan for My Sites (SharePoint 2013 Preview).
- In the Language Options section, specify whether users can select a preferred language for their My Site. The available languages correspond to the language packs installed in the farm. All servers in a farm must have the same language packs. For more information about multilingual sites, see Plan for multilingual sites (SharePoint Server 2010). For more information about language packs, see About language IDs and language packs in Install or uninstall language packs for SharePoint 2013.
- In the Read Permission Level section, specify the users or groups that can view other users‘ My Sites when they are created. By default, this includes all authenticated users. However, you can select a more specific group or users depending on the needs of your deployment.
- In the Security Trimming Options section, specify how system generated posts are checked for permissions before they are displayed in feeds and on the Tags and Notes page.
-
In the Newsfeed section, enable system generated posts to the feed on My Sites by selecting Enable activities in My Site newsfeeds. This option is selected by default. This is important in hosted environments where tenants can share the same User Profile service but have different requirements on whether they can enable newsfeeds for their users.
When upgrading from a SharePoint Server 2010 server farm that uses the newsfeed and tags and notes, you enable these legacy features on your SharePoint Server 2013 server farm by selecting Enable SharePoint 2010 activity migration.
- In the E-mail Notifications section, specify an email address to use as the sender email address for My Site email notifications. This account does not have to be a real monitored email address. If you want to receive notifications for newsfeed activities, such as replies to your posts or when someone follows you, select Enable newsfeed email notifications.
Important:
You must add the IP address of the farm’s outbound SMTP server to the safe list in Exchange Server 2013 to prevent My Site email notifications from being sent to the Junk folder. For more information about safe lists in Exchange Server 2013, see Understanding Connection Filtering in the Exchange Server Technical Library.
- In the My Site Cleanup section, specify a new owner of a My Site if the existing My Site user is removed from the profile database. For example, if a user leaves the company and is no longer in the profile database, the user‘s My Site will be deleted together with any content. However, before it is deleted, a new owner can recover any important content. Select Enable access delegation for the My Site cleanup job to first attempt to assign ownership of the My Site to the user‘s manager. If no manager is found, the My Site is assigned to the user specified in Secondary Owner. The new owner has two weeks to retrieve content from the My Site before it is deleted.
- In the Privacy Settings section, select Make My Sites Public to make all users’ My Sites public. This option is not selected by default.
Note:
When a user’s My Site is public, the user’s list of followers, the user’s list of people they are following, and all activities (including new follow notifications, social tagging and rating of content, birthdays, job title changes, workplace anniversary, updating Ask Me About, posting on a note board, and new blog posts) will be public. Any policies set within People and Privacy on the Manage Policies page is overridden.
- Click OK.
For more information about additional timer jobs for My Sites, see Planning for jobs and schedules in Plan for My Sites (SharePoint 2013 Preview).
-
Enable the User Profile Service Application – Activity Feed Job
The User Profile Service Application – Activity Feed Job creates system generated posts in the feeds for the following events:
- Following a tag
- Tagging an item
- Birthday celebration
- Job title change
- Workplace anniversary
- Updates to Ask Me About
- Posting on a note board
After you configure My Sites, enable the User Profile Service Application – Activity Feed Job so that users receive system generated posts in the Newsfeed on their My Sites.
There are other timer jobs related to My Sites that you might want to review and change default settings for. For more information about jobs related to My Sites functionality, see Planning for jobs and schedules in Plan for My Sites (SharePoint 2013 Preview).
To enable the User Profile Service Application – Activity Feed Job
- Following a tag
- Verify that you have the following administrative credentials:
- To configure timer jobs, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website.
- In Central Administration, click Monitoring, and then click Review job definitions.
- On the Job Definitions page, in the View list, select Service. The Service list appears.
- If the Service list does not display User Profile Service, in Service, click No selection, then click Change Service. On the Select Service page, use the arrows in the upper-right corner to locate User Profile Service, and then click it. The Job Definitions page updates with the User Profile service jobs.
- Click the activity feed job for the User Profile service application that you created in Prerequisites earlier in this article. The job name is in the format User_Profile_service_name – Activity Feed Job, where User_Profile_service_name is the name that you specified for your User Profile service application.
- On the Edit Timer Job page, in the Recurring Schedule section, select the interval that you want the job to run. Available intervals are Minutes, Hourly, Daily, Weekly, and Monthly. Selecting a shorter interval, such as Minutes or Hourly, ensures that activities appear on users’ My Site newsfeeds more frequently. However, it increases load on the system depending on how many activities are available. Selecting a longer interval, such as Daily, Weekly, or Monthly, reduces the number of times the job runs and processes feeds. However, it also means that users receive less frequent updates to activities in their newsfeeds.
- Click Enable.
- Optionally, click Run Now to run the job immediately without waiting for the next scheduled interval.
-
Next steps
After you configure My Sites by using the procedures in this article, consider whether you require the following optional procedures:
- Configure trusted My Site host locations
- Configure links to Office client applications
- Add personalization site links on My Sites
- Start related services
Trusted My Site Host Locations is an optional feature that prevents a user from creating more than one My Site in an organization with multiple User Profile service applications. For more information, see Add or delete a trusted My Site host location (SharePoint Server 2010).
Users‘ My Sites are convenient locations for users to save files that they work on in Office client applications, such as Word, Excel, and PowerPoint. After you configure an environment for My Sites, you can add a link to the Favorite Links section that users see when they save documents in the Save As dialog box in Office client applications. Users can then select their My Site and save files to the Documents library available on their My Site. For more information, see Add or delete links to Office client applications (SharePoint Server 2010).
If your organization wants to provide important information to users, it can do so by adding personalization site links to a user’s My Site. For more information, see Add or delete personalization site links on My Sites.
If the related services for My Sites have not been started yet, start them so that My Sites functionality is available in your environment. For more information, see Manage services on the server (SharePoint Server 2010).
Setting up My Sites is the first step in configuring microblog features in SharePoint Server 2013. For more information about how to configure microblogging features in SharePoint Server 2013, see Configure microblogging in SharePoint Server 2013.
- Configure trusted My Site host locations
-
Create and configure communities in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to create Community Sites and Community Portals in SharePoint Server 2013.
Applies to: SharePoint Server 2013
You can create Community Sites and Community Portals in SharePoint Server 2013. Community Sites provide a discussion forum experience in the SharePoint environment. The Community Portal provides a directory of Community Sites for users to browse and search for communities of interest. Before you create Community Sites and Community Portals, understand the concepts and planning process in Communities overview (SharePoint 2013 Preview) and Plan for Communities (SharePoint 2013 Preview).
Important:
The steps in this article apply to SharePoint Server 2013.
In this article:
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products
- Keyboard shortcuts
- Touch
Before you begin this operation, review the following information about prerequisites:
- Verify that you have met the requirements in Phase 3: Plan the solution in Plan for Communities (SharePoint 2013 Preview).
- Plan browser support
-
Create a Community Site
Use the following procedure to create a Community Site at the site collection level in SharePoint Server 2013.
To create a Community Site
- Verify that you have the following administrative credentials:
- To create a site collection by using the Community Site template, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website or a service application administrator. If you are a service application administrator, you must also have permission to create site collections in the web application in which you create the Community Site.
- In Central Administration, click Application Management, and then click Create site collections.
- On the Create Site Collection page, in the Web Application section, ensure that the selected web application is the web application in which you want to create the Community Site. If it is not, expand the list, and then click Change Web Application. In the Select Web Application dialog box, select a different web application.
- In the Title and Description section, type a title and description for the site collection.
- In the Web Site Address section, select the URL where you want this site collection created.
- In the Template Selection section, in the Select experience version list, select 2013. Then, on the Collaboration tab, click Community Site.
- In the Primary Site Collection Administrator section, and optionally in the Secondary Site Collection Administrator section, type an account in the format domain\username to specify an administrator for the site collection.
- Optionally, in the Quota Template section, select a quota template.
- Click OK.
- Verification: After the site collection is created successfully, click the link to open the Community Site.
-
Create a Community Portal
Use the following procedure to create a Community Portal in SharePoint Server 2013. Community Portals can be created at only the site collection level.
To create a Community Portal
- Verify that you have the following administrative credentials:
- To create a site collection by using the Community Portal template, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website or a service application administrator. If you are a service application administrator, you must also have permission to create site collections in the web application in which you create the Community Portal.
- In Central Administration, click Application Management, and then click Create site collections.
- On the Create Site Collection page, in the Web Application section, ensure that the selected web application is the web application in which you want to create the Community Portal. If it is not, expand the list, and then click Change Web Application. In the Select Web Application dialog box, select a different web application.
- In the Title and Description section, type a title and description for the site collection.
- In the Web Site Address section, select the URL where you want this site collection created.
- In the Template Selection section, in the Select experience version list, select 2013. Then, on the Enterprise tab, click Community Portal.
- In the Primary Site Collection Administrator section, and optionally in the Secondary Site Collection Administrator section, type an account in the format domain\username to specify an administrator for the site collection.
- Optionally, in the Quota Template section, select a quota template.
- Click OK.
- Verification: After the site collection is created successfully, click the link to open the Community Portal.
-
Additional steps
After you have created a Community Site or a Community Portal, consider the following additional steps to complete the configuration:
- Create additional Community Sites as needed. You might create them at the site collection level, as in this procedure, or create them at the site level depending on what you determined during the planning phase.
- Configure permissions for your Community Sites to make them private, closed, or open. For more information, see Community types in Plan for Communities (SharePoint 2013 Preview).
- Customize the Community Site. In particular, update the icon, title, and description of the site so that the Community Portal displays distinct information for each Community Site.
- Run a search crawl so that it indexes the new site or sites, and populates the Community Portal with Community Sites. No communities appear on the portal until you run a crawl. Configure the incremental crawl schedule so that the Community Portal continues to display any new Community Sites, and so that members can search within communities and the portal.
- Create additional Community Sites as needed. You might create them at the site collection level, as in this procedure, or create them at the site level depending on what you determined during the planning phase.
-
Configure microblogging in SharePoint Server 2013
Published: July 16, 2012
Summary: Use these TechNet articles to learn how to configure microblogging in SharePoint 2013.
Applies to: SharePoint Server 2013
The following articles on TechNet provide information about microblogging in SharePoint 2013. Before you configure microblogging, make sure that you have completed the steps in Configure My Sites in SharePoint Server 2013.
-
TechNet articles about microblogging
The following articles about microblogging are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Learn how to configure Following settings for My Sites in SharePoint 2013.
Manage Feed Cache and Last Modified Time Cache repopulation in SharePoint Server 2013
Learn how to manage repopulation of the Feed Cache and Last Modified Time Cache in SharePoint 2013.
Manage the Distributed Cache service in SharePoint Server 2013
Learn how to configure and manage the Distributed Cache service in SharePoint 2013.
-
Configure Following settings in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to configure Following settings for My Sites in SharePoint Server 2013.
Applies to: SharePoint Server 2013
In SharePoint Server 2013, following is a user-initiated action that indicates the user’s interest in a specific document, person, site, or tag. When users follow an item, new activities about that item appear in the users‘ newsfeeds on their My Sites. Users view all their followed items from their My Sites.
-
Configure Following settings for My Sites
Use this procedure to configure Following settings for My Sites.
Note:
Using lower limits can slightly improve performance. Also, by using lower limits, users will follow higher priority documents, people, or sites.
To configure Following settings for My Sites
- Verify that you have the following administrative credentials:
- To configure Following settings for the User Profile service application, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website or a service application administrator for the User Profile service application.
- In Central Administration, in the Application Management section, in the Service Applications group, click Manage service applications.
- In the list of service applications, select the User Profile service application.
- In the Operations group, click Manage.
- On the Manage Profile Service page, in the My Sites Settings section, click Manage Following.
- In the Maximum number of followed people box, type the maximum number of people that a user can follow from the user‘s My Site.
- In the Maximum number of followed documents box, type the maximum number of documents that a user can follow from the user‘s My Site.
- In the Maximum number of followed sites box, type the maximum number of sites that a user can follow from the user‘s My Site.
- Click OK.
-
Manage Feed Cache and Last Modified Time Cache repopulation in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to manage repopulation of the Feed Cache and Last Modified Time Cache in SharePoint Server 2013.
Applies to: SharePoint Server 2013
SharePoint Server 2013 feeds require the Feed Cache and Last Modified Time Cache. The Feed Cache maintains recent conversations and activities of entities. The Last Modified Time Cache maintains the last modified time for all items in the Feed Cache. The Distributed Cache service manages both the Feed Cache and the Last Modified Time Cache.
System events, such as a server shutting down unexpectedly or a variation in electrical supply to the server, can affect the Distributed Cache service. Additionally, an administrator who performs maintenance and operational tasks can take an application server that runs the Distributed Cache service offline. This results in the resetting and emptying of the Feed Cache and the Last Modified Time Cache. In this situation, repopulation of the recent conversations and activities of entities occurs. Repopulation occurs in two stages:
- Load last modified time information for recent conversations and activities.
- Load recent conversations and activities.
Note:
In the case of planned maintenance and operations, an administrator can preserve cache data by using the graceful shutdown procedure. For more information, see Perform a graceful shutdown of the Distributed Cache service in Manage the Distributed Cache service in SharePoint Server 2013.
To manage the repopulation process, SharePoint Server 2013 includes the Feed Cache Repopulation Job timer job. When the Feed Cache Repopulation Job timer job runs, it first checks whether the Feed Cache and Last Modified Time Cache are empty. If they are empty, it starts repopulating the last modified time information for recent conversations and activities in the Last Modified Time Cache. After the timer job finishes the Last Modified Time Cache repopulation, the Feed Cache is populated with recent conversations and activities the next time any user accesses a feed in SharePoint Server 2013.
In this article:
- Repopulate the Last Modified Time Cache by using timer jobs in Central Administration
- Repopulate the Feed Cache and Last Modified Time Cache by using Windows PowerShell cmdlets
-
Repopulate the Last Modified Time Cache by using timer jobs in Central Administration
The User Profile Service Application – Feed Cache Repopulation Job repopulates the Last Modified Time Cache if the Distributed Cache service resets and becomes empty. Also, after you configure My Sites, users will not see posts appearing in their consolidated newsfeed if the User Profile Service Application – Feed Cache Repopulation Job timer job is not configured to run. By default, the User Profile Service Application – Feed Cache Repopulation Job timer job is configured to run every 5 minutes.
Use this procedure to configure the User Profile Service Application – Feed Cache Repopulation Job timer job to monitor the Feed Cache and Last Modified Time Cache for repopulation.
Important:
Do not change the default settings of this timer job if you plan to use social features in SharePoint Server 2013. Do not disable this timer job. If this timer job is disabled and a repopulation is required, it will re-enable itself and run.
To configure the User Profile Service Application – Feed Cache Repopulation Job
- Verify that you have the following administrative credentials:
- To configure timer jobs, you must be a member of the Farm Administrators group on the computer running the SharePoint Central Administration website.
- In Central Administration, on the Monitoring page, click Review job definitions.
- On the Job Definitions page, in the View list, select All.
- Use the arrows at the bottom of the page to locate the feed cache repopulation job for the User Profile service application on your server farm. The job name is in the format User_Profile_service_name – Feed Cache Repopulation Job, where User_Profile_service_name is the name that you specified for the User Profile service application.
- On the Edit Timer Job page, in the Recurring Schedule section, select the interval that you want the job to run. Available intervals are Minutes, Hourly, Daily, Weekly, and Monthly. Selecting a shorter interval, such as Minutes or Hourly, ensures that checks for an empty cache is performed more frequently. Selecting a longer interval, such as Daily, Weekly, or Monthly, reduces the number of times the job runs. However, it also means that performing cache repopulation checks are done fewer times. We recommend that this timer job runs on shorter intervals.
- Click Enable.
- Optionally, click Run Now to run the job immediately without waiting for the next scheduled interval.
-
Repopulate the Feed Cache and Last Modified Time Cache by using Windows PowerShell cmdlets
You can use Windows PowerShell cmdlets to perform the repopulation of the Feed Cache and the Last Modified Time Cache. To perform repopulation, we recommend that you configure the User Profile Service Application – Feed Cache Repopulation Job timer job as described above. This is because the timer job first checks to see whether the cache is empty and then repopulates the cache as necessary, whereas the Windows PowerShell cmdlets force a repopulation of the cache. However, in some instances, using the Windows PowerShell cmdlets is the only way to repopulate the cache. These instances include the following:
- After attaching a new content database and the cache is not repopulating.
- After restoring a content database.
The following cmdlets are available to force repopulation of the Feed Cache and the Last Modified Time Cache:
- Update-SPRepopulateMicroblogLMTCache. This cmdlet must be run first.
- Update-SPRepopulateMicroblogFeedCache.
- After attaching a new content database and the cache is not repopulating.
-
Manage the Distributed Cache service in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to configure and manage the Distributed Cache service in SharePoint Server 2013.
Applies to: SharePoint Server 2013
To perform management and operational tasks on the Distributed Cache service in SharePoint Server 2013, an administrator must perform specific, ordered procedures. This article describes how to conduct several management and operational tasks on the Distributed Cache service.
In this article:
- Start and stop the Distributed Cache service
- Change the memory allocation of the Distributed Cache service
- Add or remove a server in a Distributed Cache cluster
- Perform a graceful shutdown of the Distributed Cache service
- Change the service account
Important:
The Distributed Cache service can end up in a nonfunctioning or unrecoverable state if you do not follow the procedures that are listed in this article. In extreme scenarios, you might have to rebuild the server farm. The Distributed Cache depends on Windows Server AppFabric as a prerequisite. Do not administer the AppFabric Caching Service from the Services window in Administrative Tools in Control Panel. Do not use the applications in the folder named AppFabric for Windows Server on the Start menu.
- Start and stop the Distributed Cache service
-
Start and stop the Distributed Cache service
An administrator that performs maintenance and operational tasks might need to start and stop the Distributed Cache service. Some of these tasks include the following:
- Changing the default configuration of the server farm at installation time. The Distributed Cache service is started on all SharePoint servers at installation time. An administrator might want to stop the Distributed Cache service on some servers in the farm.
- Updating the server and there is only one Distributed Cache server in the SharePoint Server 2013 farm.
Stopping the cache results in partial data loss. The Feed Cache depends on the Distributed Cache service. Tags and document activities are saved only to the Feed Cache. Tags and document activities are not persisted to content databases. When the Distributed Cache service is stopped, tags and document activities are lost. When the Distributed Cache service is started, repopulation occurs when the feed cache repopulation timer job runs. For more information, see Manage Feed Cache and Last Modified Time Cache repopulation in SharePoint Server 2013. One way to maintain the tags and document activities is to use the method described in Perform a graceful shutdown of the Distributed Cache service later in this article. When the graceful shutdown of the Distributed Cache service method is used, all cache data is moved from one server to another server before the Distributed Cache service is stopped.
Note:
If your cache hosts are part of a cache cluster, do not start or stop the Distributed Cache service as described here. Instead, see Add or remove a server in a Distributed Cache cluster later in this article.
To start and stop the Distributed Cache service by using Central Administration
- Changing the default configuration of the server farm at installation time. The Distributed Cache service is started on all SharePoint servers at installation time. An administrator might want to stop the Distributed Cache service on some servers in the farm.
- In Central Administration, click Application Management.
- In Service Applications, click Manage Services on Server.
- On the Services on Server page, locate the Distributed Cache service.
- If the Distributed Cache service is started and you want to stop the service, under Action, click Stop. If the Distributed Cache service is stopped and you want to start the service, under Action, click Start.
To start the Distributed Cache service by using Windows PowerShell
At the Windows PowerShell command prompt, run the following command:
$instanceName =”SPDistributedCacheService Name=AppFabricCachingService”
$serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername}
$serviceInstance.Provision()
To stop the Distributed Cache service by using Windows PowerShell
At the Windows PowerShell command prompt, run the following command:
$instanceName =”SPDistributedCacheService Name=AppFabricCachingService”
$serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername}
$serviceInstance.Unprovision()
-
Change the memory allocation of the Distributed Cache service
When SharePoint Server 2013 is installed, it assigns the Distributed Cache service 10 percent of the total physical memory on the server. The Distributed Cache service uses half of the memory allocation for data storage (also known as cache size), and the other half of the memory allocation is used for memory management overhead. When the cached data grows, the Distributed Cache service uses the entire 10 percent of the allocated memory.
The following scenarios describe when an administrator should increase the memory allocation for the Distributed Cache:
- When you add physical memory to a server. In this case, the Distributed Cache does not recalculate the 10 percent memory allocation to include the new total physical memory.
- When you have a dedicated Distributed Cache server. Use the following method to calculate how much memory can be assigned to the Distributed Cache service:
- When you add physical memory to a server. In this case, the Distributed Cache does not recalculate the 10 percent memory allocation to include the new total physical memory.
- Determine the total physical memory on the server. For this example, we will use 16 GB as the total physical memory available on the server.
- Reserve 2 GB of memory for other processes and services that are running on the cache host. For example, 16 GB – 2 GB = 14 GB. This remaining memory is allocated to the Distributed Cache service.
- Take half of the remaining memory, and convert it to MB. For example, 14 GB/2 = 7 GB or 7000 MB. This is the cache size of the Distributed Cache service.
- Use the following procedure to update the memory allocation accordingly.
-
Change the memory allocation of the Distributed Cache by using Windows PowerShell
Use this procedure to reconfigure the memory allocation for the Distributed Cache service.
-
Stop the Distributed Cache service on all cache hosts that are part of the cache cluster. To stop the Distributed Cache service, on all cache hosts, at the Windows PowerShell command prompt, run the following command:
$instanceName =”SPDistributedCacheService Name=AppFabricCachingService”
$serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername}
$serviceInstance.Unprovision()
-
Reconfigure the cache size of the Distributed Cache service on the server that is being added or upgraded. On that server only, at the Windows PowerShell command prompt, run the following command:
Set-CacheHostConfig -Hostname Hostname -cacheport Cacheport -cachesize Cachesize
Where:
- Hostname is the FQDN of the application server being reconfigured that runs the Distributed Cache service.
- Cacheport is equal to the port number of the Distributed Cache (22233).
- Cachesize is the cache size’s memory allocation assignment in MB. In the previous example, the cache size was calculated at 7000 MB for a server with 16 GB of total physical memory.
-
Restart the Distributed Cache service. On all servers, at the Windows PowerShell command prompt, run the following command:
$serviceInstance.Provision()
-
Add or remove a server in a Distributed Cache cluster
An administrator can add or remove a server to a cache cluster, or might want to remove a server from the cache cluster, perform some operational or maintenance tasks on the server, and then rejoin or add the server to the cache cluster. When removing the server, the Distributed Cache service is stopped, then unregistered from the server. Unregistering the Distributed Cache service means that an administrator will not see the Distributed Cache service listed on the Services on Server page in Central Administration. Similarly, when a server is added, the Distributed Cache service is registered and then is started on the server. Registering the Distributed Cache service means that an administrator will see the Distributed Cache service listed on the Services on Server page in Central Administration.
Use the following procedures to add and remove a server from a cache cluster. These Windows PowerShell cmdlets are run on the server being added or removed.
-
Add a server to the cache cluster and starting the Distributed Cache service by using a Windows PowerShell
At the Windows PowerShell command prompt, run the following command:
Add-SPDistributedCacheServiceInstanceOnLocalServer
-
Remove a server from the cache cluster by using a Windows PowerShell
At the Windows PowerShell command prompt, run the following command:
Remove-SPDistributedCacheServiceInstanceOnLocalServer
Important:
This procedure will stop the cache service and nonpersisted cached data will be lost. If you want to keep the cached data, use the graceful shutdown procedure that is described in the next section, and then run the Remove-SPDistributedCacheServiceInstanceOnLocalServer cmdlet. The Remove-SPDistributedCacheServiceInstanceOnLocalServer cmdlet involves stopping and disabling the underlying AppFabric Caching service. Do not restart the AppFabric Caching service other than by running the Add-SPDistributedCacheServiceInstanceOnLocalServer cmdlet.
-
-
Perform a graceful shutdown of the Distributed Cache service
In a SharePoint Server 2013 farm, a cache cluster exists when several cache hosts run the Distributed Cache service. In a SharePoint Server 2013 farm, one cache exists, and the cache spans the cache cluster. An administrator can take a cache host out of the cluster to perform operational or maintenance tasks on the server, such as applying updates to the server. To prevent data loss associated with the removal of the cache host from the cache cluster, an administrator must first run the graceful shutdown procedure before removing the cache host from the cache cluster. The graceful shutdown procedure is run on the cache host being removed from the cache cluster. This cache host stores a portion of the cached data. The graceful shutdown procedure transfers all cached data from the cache host on which the graceful shutdown procedure is being run on to another cache host in the farm. The transfer process takes 15 minutes or more to run depending on how many items exist in the cache. When the transfer process is complete, removing the cache host by using the Remove-SPDistributedCacheServiceInstanceOnLocalServer cmdlet does not result in any data loss.
To perform a graceful shutdown of the Distributed Cache by using Windows PowerShell
At the Windows PowerShell command prompt, run the following command:
Stop-SPDistributedCacheServiceInstanceGracefullyOnLocalServer
Remove-SPDistributedCacheServiceInstanceOnLocalServer
Note:
To rejoin or add the server to the cache cluster, run the Add-SPDistributedCacheServiceInstanceOnLocalServer cmdlet.
-
Change the service account
When the server farm is first configured, the server farm account is set as the service account of the AppFabric Caching service. The Distributed Cache service depends on the AppFabric Caching service. To change the service account of the AppFabric Caching service to a managed account:
- Create a managed account. For more information, see Configure automatic password change (SharePoint Server 2010).
-
Set the Managed account as the service account on the AppFabric Caching service. At the Windows PowerShell command prompt, run the following command:
$farm = Get-SPFarm
$cacheService = $farm.Services | where {$_.Name -eq “AppFabricCachingService”}
$accnt = Get-SPManagedAccount -Identity domain_name\user_name
$cacheService.ProcessIdentity.CurrentIdentityType = “SpecificUser”
$cacheService.ProcessIdentity.ManagedAccount = $accnt
$cacheService.ProcessIdentity.Update()
$cacheService.ProcessIdentity.Deploy()
Where Domain_name\user_name is the domain name and user name of the managed account.
-
Enable or disable personal and social features for users or groups in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how to configure user permissions for personal and social features in SharePoint Server 2013.
Applies to: SharePoint Server 2013
Farm Administrators or service administrators of a User Profile service application control who can create My Sites, and use personal and social features. For example, you might want a subset of users in an organization to be able to create My Sites, so you enable the Create Personal Site permission for those users. For more information, see User Profile service application overview (SharePoint 2013 Preview).
Important:
Before you decide which permission to grant users or groups of users, first review the permission descriptions and how the combination of these permissions affects the My Site user experience. For more information, see Plan users and user permissions in Plan for My Sites (SharePoint 2013 Preview).
Before you perform this procedure, confirm the following:
- A User Profile service application is running in the farm. For more information, see Create, edit, or delete a User Profile service application (SharePoint 2013 Preview).
- A User Profile service application is running in the farm. For more information, see Create, edit, or delete a User Profile service application (SharePoint 2013 Preview).
-
Enable users or groups to use personal and social features
Use this procedure to configure the user permissions for personal and social features.
To enable users or groups to use personal and social features
- Verify that you have the following administrative credentials:
- To use the SharePoint Central Administration website to enable users or groups to use personal and social features, you must be a member of the Farm Administrators group, or you must have been delegated permission to administer the User Profile service application that is running in the farm. For more information, see Assign administration of a User Profile service application (SharePoint 2013 Preview).
- In Central Administration, in the Application Management section, click Manage service applications.
- In the list of service applications, click User Profile Service Application.
- On the Manage Profile Service: User Profile Service Application page, in the People group, click Manage User Permissions.
- On the Permissions for User Profile Service Application page, type or select a user or group account, and then click Add.
- In the Permissions for box, check the feature or features that you want the user or group to be able to use, and then click OK.
-
Configure web content management solutions in SharePoint Server 2013
Updated: October 16, 2012
Summary: Learn how to install and configure SharePoint web content management solutions that use cross-site collection publishing.
Applies to: SharePoint Server 2013
-
The articles that are listed in the following table describe how to set up cross-site publishing features in a SharePoint Server 2013 environment.
Content
Description
Learn how to create site collections for cross-site publishing, activate the Cross-Site Collection Publishing feature, create and manage term sets for tagging content on authoring sites, create catalog content by using SharePoint lists, share a library or list as a catalog, and configure search settings for cross-site publishing.
Connect a publishing site to a catalog in SharePoint Server 2013
Learn how to connect a publishing site to a library or list that is shared as a catalog.
Learn how to configure the following Web Parts that use search technology in a publishing environment:
- Content Search Web Part
- Refinement Panel Web Part
- Taxonomy Refinement Panel Web Part
- Recommended Items Web Part
Configure refiners and faceted navigation in SharePoint Server 2013
Learn how to map a crawled property to a refinable managed property, enable a managed property as a refiner, and configure faceted navigation.
Configure result sources for web content management in SharePoint Server 2013
Learn how to create and manage result sources for SharePoint Search service applications, and for SharePoint sites and site collections.
Configure recommendations and usage event types in SharePoint Server 2013
Learn how to create custom usage event types, how to add code to record usage events, and how to influence how recommendations are shown on a page.
- Content Search Web Part
-
Configure cross-site publishing in SharePoint Server 2013
Updated: October 16, 2012
Summary: Learn to create and tag catalog content in authoring sites and configure search settings for cross-site publishing in SharePoint Server 2013.
Applies to: SharePoint Server 2013
Before you configure cross-site publishing, make sure that you understand the concepts and terminology in Plan for cross-site publishing in SharePoint 2013 Preview.
In this article:
- Before you begin
- Create site collections for cross-site publishing
- Activate the Cross-Site Collection Publishing feature
- Create and manage term sets for tagging content on authoring sites
- Create catalog content by using SharePoint lists
- Share a library or list as a catalog
- Make a term set available to other site collections
- Configure search for cross-site publishing
- Before you begin
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Create site collections for cross-site publishing
In a cross-site collection publishing scenario where content is reused across site collections, you must have at least two site collections, one for authoring content and one for publishing content. Before you create the site collections, review the following information:
- “Plan site collections and site structure for SharePoint authoring sites” in Plan SharePoint authoring sites for cross-site publishing (SharePoint 2013 Preview).
- “Plan site collections and site structure for SharePoint publishing sites” in Plan SharePoint publishing sites for cross-site publishing (SharePoint 2013 Preview).
For information about how to create a site collection by using either Central Administration or Windows PowerShell, see Create a site collection (SharePoint 2013 Preview).
- “Plan site collections and site structure for SharePoint authoring sites” in Plan SharePoint authoring sites for cross-site publishing (SharePoint 2013 Preview).
-
Activate the Cross-Site Collection Publishing feature
Before you can use cross-site collection publishing to reuse content across site collections, you have to activate the Cross-Site Collection Publishing feature on the authoring site collection.
Note:
If you used the Product Catalog Site Collection template to create the authoring site collection, you do not have to do this operation. By default, the Cross-Site Collection publishing feature is active when you create a site collection by using the Product Catalog Site Collection template.
To activate the Cross-Site Collection Publishing feature
- Verify that the user account that performs this procedure is a site collection administrator on the authoring site collection.
- On the top-level site of the authoring site collection, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Collection Administration section, click Site collection features.
- On the Site Collection Features page, next to Cross-Site Collection Publishing, click Activate.
-
Create content for authoring sites
Before you create content for authoring sites, review “Plan term sets for tagging content on authoring sites” and “Plan catalog content for authoring sites” in Plan SharePoint authoring sites for cross-site publishing (SharePoint 2013 Preview).
You create and manage term sets by using the Term Store Management Tool. For information about how to create and manage term sets, see the following articles:
After you have created a term set, you have to make it available for tagging content. If you used the Product Catalog Site Collection template to create the authoring site collection, and you have created a term set in this site collection, you do not have to do this operation. By default, new term sets created in the Product Catalog site collection are available for tagging content.
To make a term set available for tagging content
- Verify that the user account that performs this procedure is a member of the Owners SharePoint group on the authoring site that contains the catalog.
- On the authoring site, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Administration section, click Term store management.
- In the TAXONOMY TERM STORE section, click the term set that you want to make available for tagging.
- Click the INTEDED USE tab, and then select Available for Tagging.
- Click Save.
When you create catalog content by using SharePoint lists, we recommend that you create site columns for the lists in which you want to maintain your catalog content. This is because managed properties are automatically created for site columns, and you can use these managed properties when defining queries for you catalog content on a publishing site. If you have several lists, we recommend that you create a site content type for each list, and then associate the appropriate site columns to this site content type. If you want to use managed navigation to display catalog content on a publishing site, you also have to create at least one term set as described in Create and manage term sets for tagging content on authoring sites. The tagging term set must be tied to a site column that is a Managed Metadata data type.
For information about how to create site content types and site columns, see the following articles:
If you have large amounts of data in external business systems — for example, an ERP system — consider importing this data into one or more SharePoint lists. SharePoint Server 2013 does not have a solution for importing list content. However, you can develop custom import tools — for example, by using Windows PowerShell. For a set of example Windows PowerShell scripts that you can use to import list content for cross-site publishing, see Import list content to Products list for SharePoint 2013 Preview. The example scripts import content only to a site collection that was created by using the Product Catalog Site Collection template.
Before you share a library or list as a catalog, verify that the Cross-Site Collection Publishing feature is activated for the site collection. If you used the Product Catalog Site Collection template to create the site collection, the Cross-Site Collection Publishing feature is already active. For all other types of site collections, you must activate the Cross-Site Collection Publishing feature before you can continue with the following steps. For more information, see Activate the Cross-Site Collection Publishing feature earlier in this article.
By default, anonymous access is enabled when you share a library or list as a catalog. If you have connected a publishing site to the catalog, and you don’t want anonymous users to be able to view and search content that was added to the search index from this catalog, you should disable anonymous access.
Important:
In addition to enabling anonymous access for a catalog, you must enable anonymous access for the web application and publishing site so that anonymous users can search and view the content. For more information, see Create claims-based web applications in SharePoint 2013.
To share a library or list as a catalog
- Verify that the user account that performs this procedure is a member of the Owners group on the site that contains the library or list that you want to share.
- Browse to the library or list that you want to share, and then do one of the following:
- To share a library, click the LIBRARY tab, and then, on the ribbon, in the Settings group, click Library Settings.
- To share a list, click the LIST tab, and then, on the ribbon, in the Settings group, click List Settings.
- On the Settings page, in the General Settings section, click Catalog Settings.
- On the Catalog Settings page, in the Catalog Sharing section, select the Enable this library as a catalog check box.
- In the Anonymous Access section, if you want don’t want anonymous users to view and search this content, click Disable anonymous access.
-
In the Catalog Item URL Fields section, in the Available fields box, select up to five fields that uniquely identify an item in the library or list, and then click Add.
After you connect a publishing site to this catalog, the fields that you specified as catalog item URL fields appear as part of the friendly URL. (See the example that follows this procedure.)
- In the Navigation Hierarchy section, select the column that is associated with the term set that you want to use as a navigation term set for catalog pages. After you connect a publishing site to this library or list to show catalog content, the value of the column that you selected appears as part of the friendly URL (see the example that follows this procedure).
Note:
You only have to make a selection in this section if you want to use managed navigation to display catalog content on a publishing site.
- Click OK.
Note:
After you share a library or list as a catalog, the content source that contains the catalog must be crawled. You don’t have to start a full crawl. This is because an incremental crawl or a continuous crawl also adds the content to the search index. For more information, see Start, pause, resume, or stop crawls in SharePoint 2013 Preview.
In this example, let’s say that you have a list that contains data for different electronic products. The following items were specified when the list was shared as catalog:
-
Electronic products
- Audio
- Car audio
- MP3
- Computers
- Laptops
- Desktops
- Audio
Each item in the shared list is associated with a value from this term set in the Item Category Managed Metadata site column. For more information about Managed Metadata columns, see Create a Managed Metadata column.
The following table describes how site columns and their corresponding values in the previous list are combined to create friendly URLs for catalog content when you connect a publishing site collection to this list.
|
Product title |
Item Category |
Item Number |
Friendly URL to an item when the catalog is connected to a publishing site |
|
Proseware 50W Car Radio |
Car audio |
1010101 |
<site>/audio/car-audio/1010101 |
|
Contoso 4GB Portable MP3 Player M450 |
MP3 |
4020102 |
<site>/audio/mp3/4020102 |
|
AdventureWorks Laptop8.9 E0890 |
Laptops |
7030906 |
<site>/computers/laptops/7030906 |
|
WWI Desktop PC2.33 X2330 |
Desktops |
7030906 |
<site>/computers/desktops/3030802 |
After you create a term set on the authoring site collection, you have to make it available to publishing site collections. You can make a term set available to all site collections or to specific site collections.
To make a term set available to all site collections
- Verify that the user account that performs this procedure is a member of the Owners SharePoint group on the authoring site that contains the catalog.
- On the authoring site, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Administration section, click Term store management. If the user that performs this procedure is already a member of the Term Store Administrators group, you can skip to step 7.
- In the Term Store Management Tool, verify that Managed Metadata Service is selected.
- In the Term Store Administrator section, type one or more user names.
- Click Save.
- Right-click Managed Metadata Service, and then select New Group.
- Type the name of the global term set that you want to create, and then press Enter.
- Refresh the page.
- Right-click the term set that you want to make available to all site collections, and then click Move Term Set.
- In the Term Set Move dialog box, click the global term set that you want to move the term set to, and then click OK.
- Refresh the page.
To make a term set available to specific site collections
- Verify that the user account that performs this procedure is a member of the Owners SharePoint group on the authoring site that contains the catalog.
- On the authoring site, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Administration section, click Term store management.
- In the Term Store Management Tool, click the group that contains all term sets within the site collection.
- In the Site Collection Access section, type the URLs of the site collections to which you want to make the term set available — for example, http://<site>/sites/products.
- Click Save.
-
Configure search for cross-site publishing
Because cross-site publishing depends on search, you have to create a content source and manage crawling for SharePoint cross-site publishing sites.
A content source specifies what, when, and how content should be crawled. When a Search service application is created, a content source named Local SharePoint sites is created and is automatically configured to crawl all SharePoint sites in the local server farm. You can create additional content sources to specify other content to crawl and define how SharePoint should crawl that content. You do not have to create a separate content source for catalog content in order to make content available to other site collections. However, it is easier to maintain crawl schedules when you have separate content sources for the different content that you want users to view and search.
The ability to enable continuous crawls is a new crawl schedule option in SharePoint 2013. When you enable continuous crawls, any changes that are made to content within the specified content source is picked up automatically by the crawler and added to the search index. A continuous crawl starts at set intervals. The default interval is 15 minutes, but you can set continuous crawls to occur at shorter intervals by using Windows PowerShell.
For information about how to create a new content source and manage crawling in Central Administration, see the following articles:
- Add, edit, or delete a content source in SharePoint 2013 Preview
- Start, pause, resume, or stop crawls in SharePoint 2013 Preview
Some actions — for example, doing search schema management to enable refiners — require a full reindex of the content source that contains the catalog for the changes to be added to the search index. A site collection administrator can independently of the Search service application administrator indicate that a catalog should be fully reindexed during the next scheduled crawl of the catalog.
To reindex catalog content
- Add, edit, or delete a content source in SharePoint 2013 Preview
- Verify that the user account that performs this procedure is a member of the Site collection administrators group on the site that contains the catalog.
- Browse to the catalog, and then do one of the following:
- If you want to perform a full crawl of a catalog in a library, click the LIBRARY tab, and then, on the ribbon, in the Settings group, click Library Settings.
- If you want to perform a full crawl of a catalog in a list, click the LIST tab, and then, on the ribbon, in the Settings group, click List Settings.
- On the Settings page, in the General Settings section, click Advanced settings.
- On the Advanced Settings page, in the Reindex List section, click Reindex List, and then click Reindex List to confirm that you want the catalog to be reindexed during the next scheduled crawl.
- Click OK.
Note:
The full reindex of the catalog will be performed during the next scheduled crawl.
-
Connect a publishing site to a catalog in SharePoint Server 2013
Published: October 2, 2012
Summary: Learn how to connect a publishing site collection to a library or list that is shared as a catalog.
Applies to: SharePoint Server 2013
To show content from a library or list that is shared as a catalog, you must connect the publishing site collection to the catalog. When you connect a publishing site collection to a catalog, the following occurs:
- The catalog content is integrated into the publishing site collection.
- The term set used by the catalog is integrated into the term set of the publishing site collection.
- A category page and an item details page are created for the catalog pages.
- Friendly URL is created for the item details page.
- A result source is created for the catalog.
In this article:
- The catalog content is integrated into the publishing site collection.
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
Before you connect the publishing site collection to a catalog, review the information in Plan category pages and item detail pages. Also verify the following:
- The publishing site that you connect to a catalog uses managed navigation. By default, site collections that are created by using the Publishing Portal Site Collection template use managed navigation.
- The library or list was shared as a catalog, as described in Share a library or list as a catalog.
- A full crawl of the content source that contains the catalog was performed, as described in Configure search for cross-site publishing.
- The term set that is used by the catalog is available to the publishing site collection, as described in Make a term set available to other site collections.
- Plan browser support
-
Connect a publishing site to a catalog
To connect a publishing site to a catalog
- Verify that the user account that completes this procedure is a member of the Owners SharePoint group on the publishing site collection.
- On the publishing site collection, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Administration section, click Manage catalog connections.
- On the Manage catalog connections page, click Connect to a catalog. A list of available catalogs appears. Note that only catalogs that have been crawled will appear.
- On the line that contains the catalog that you want to connect to, click Connect. You can also search for a specific catalog by typing the catalog name in the search field.
- On the Catalog Source Settings page, in the Connection Integration section, do one of the following:
- To make catalog content available to the publishing site and integrate the catalog tagging term set into the publishing site navigation term set, select Integrate the catalog into my site. When you select this option, use the following steps to specify at which level the term sets should be integrated, specify the URL for the catalog item details page, and select category pages and catalog item pages.
-
To make the catalog content available to the publishing site, select Connect, but do not integrate the catalog. You should select this option if you want to use content from the library to create individual catalog item pages.
Either option creates a result source for the catalog.
- In the Navigation Hierarchy section, specify the term from which the catalog tagging term set should be integrated into the publishing site navigation term set. The catalog navigation column that you previously configured in Share a library or list as a catalog appears by default. The fields in this section are optional. Therefore, if you don’t change the fields in this section, the catalog tagging term set will be integrated from the root term. If you want to integrate the catalog tagging term set from a different term, do the following:
- Next to the Root term of hierarchy box, click Browse for a valid choice.
- In the Select: Add Terms dialog box, click the term that corresponds to the level from which you want to integrate the catalog tagging term set, click Select, and then click OK.
-
To integrate the root term that is the parent of the selected term in the publishing site navigation term set, select the Include root term in site navigation check box.
Note:
All items in the catalog must be tagged with a term from the specified catalog tagging term set. If this is not done, site navigation will not work as intended for all items.
- In the Navigation Position section, specify the term in the publishing site navigation term set where the catalog tagging term set should be integrated. Do one of the following:
- To integrate the catalog tagging term set to the root term of the publishing site navigation term set, click Add to navigation root.
-
To integrate the catalog tagging term set to a term below the root term of the publishing site navigation term set, click Select an alternate location in site navigation, and then do the following:
- Click Browse for a valid choice to display the publishing site navigation term set.
- In the Select: Add Terms dialog box, click the term that corresponds to the level from which you want to integrate the catalog tagging term set, click Select, and then click OK.
- Click Browse for a valid choice to display the publishing site navigation term set.
- If you want changes to the catalog tagging term set to be updated on the publishing site, in the Navigation Pinning section, select the Pin terms to site navigation check box. By default, this option is selected. If you clear this check box, changes made to the catalog tagging term set are not reflected on the publishing site navigation.
- In the Catalog Item URL Behavior section, specify what you want the URL of the catalog item to do by selecting one of the following options:
- To point the URL of the catalog item to an item details page, select Make URLs relative to this site. When you select this option, you have to specify a catalog item URL format as described in the next step. This also means that the content that you can display on the item details page has to come from the search index.
- To have the catalog item URL point to the item in the source catalog, select Make URLs point to source catalog. When you select this option, you do not have to specify a catalog item URL format. Note that when you select this option, anonymous users are not able to access and view the item in the source catalog.
- In the Catalog Item URL Format section, select which properties the URL of the item details page should contain by doing one of the following:
-
To use the field that you specified as Primary Key the when you shared the library or list as a catalog as described in Share a library or list as a catalog, select Use the default URL format provided by the catalog source. By default, this option is already selected.
Note:
All items in the catalog must have values for the specified field. Site navigation will not work as intended for items with missing values.
- To manually define a format for the URL, select Manually define a URL format, and then type in a URL. You should select this option only if you have created an item details page and the items in your catalog are not tagged with a term from a catalog tagging term set. Type the URL in the following format: /<Folder of item details page>/<Name of item details page>.aspx? <Managed property name>=[Managed property value] — for example, /Pages/itemdetails.aspx?TitleProperty=[Title].
-
To construct a custom URL based on catalog properties, select Construct a URL format from catalog properties, and then do the following:
- In the Available Fields list, select up to five fields, and then click Add.
- In the Available Fields list, select up to five fields, and then click Add.
Important:
Fields of site column type Number will not create a valid URL. All items in the catalog must have values for the specified fields. Site navigation will not work as intended for items with missing values.
- In the Category Page section, do one of the following:
- To have SharePoint Server 2013 automatically create a new Category page for your catalog content, click Create a new page, and then select a master page. The page will be added to the Pages library with the name Category-<catalog tagging term set name>. The page will not be published automatically.
- To use a Category page that was already created, select Use an existing page, and then specify the location of the page.
- In the Item Page section, do one of the following:
- To have SharePoint Server 2013 automatically create a new Item page for your catalog content, click Create a new page, and then select a master page. The page will be added to the Pages library with the name CatalogItem-<catalog tagging term set name>. The page will not be published automatically.
- To use an already created Item page, select Use an existing page, and specify the location of this page.
- Click OK.
-
Configure Search Web Parts in SharePoint Server 2013
Published: October 2, 2012
Summary: Learn how to configure the different Web Parts that use search technology in a publishing environment.
Applies to: SharePoint Server 2013
Web Parts that use search technology to show content in a publishing environment (referred to in this article as Search Web Parts) show content that was crawled and added to the search index, as described in “Understanding how content is added to and managed in the search index” in Overview of cross-site publishing in SharePoint 2013 Preview. These Web Parts have queries defined in them, and when users browse to a page that contains a Web Part that uses search technology, the Web Part issues the query automatically. The query result is then displayed in the Web Part. You can modify the query in the search Web Part to fit your content needs.
In this article:
- Before you begin
- Add a Content Search Web Part to a page
- Configure the query for a Content Search Web Part
- Configure the display templates for the Content Search Web Part
- Add a Refinement Web Part to a page
- Configure the Refinement Web Part
- Display refiner counts in a Refinement Web Part
- Change the refiner display name
- Configure the display templates for the Refinement Web Part
- Add a Taxonomy Refinement Panel Web Part to a page
- Configure the Taxonomy Refinement Panel Web Part
- Add a Recommended Items Web Part to a page
- Configure the Recommended Items Web Part
- Configure the display templates for the Recommended Items Web Part
- Before you begin
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
There are many Search Web Parts available in SharePoint Server 2013. These Web Parts have predefined queries, such as what type of content to search for, where to search for content, and how to show content. For information about different Search Web Parts, see “Plan to add search Web Parts to pages” in Plan SharePoint publishing sites for cross-site publishing (SharePoint 2013 Preview). Many of the Search Web Parts use result sources and have query rules that are applied to them. Result sources narrow the scope of search results that are retrieved. A query rule is a set of conditions that will cause the query to be changed in a specific way. For more information about result sources and query rules, see Plan result sources and query rules.
To customize how search results appear in Search Web Parts — for example, to show an image followed by a title in bold to the right of the image — you modify display templates. The two types of display templates that are most relevant to Search Web Parts are control display templates and item display templates.
- Plan browser support
-
Add a Content Search Web Part to a page
To add a Content Search Web Part to a page
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you want to add the Web Part.
- Click the Settings menu, and then click Edit page.
- In the Web Part Zone where you want to add the Web Part, click Add a Web Part.
- In the Categories list, click Content Rollup.
- In the Parts list, click Content Search, and then click Add.
-
Configure the query for a Content Search Web Part
You can use the Content Search Web Part in Quick Mode and create a query by selecting options from a list of existing result sources, or you can switch to Advanced Mode to create your own custom query by using Keyword Query Language (KQL). Use the Advanced Mode only if you know KQL and the functionality that is enabled for the managed properties.
To configure the query for a Content Search Web Part
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page that contains the Content Search Web Part that you want to configure.
- Click the Settings menu, and then click Edit Page.
- In the Web Part, click the Content Search Web Part Menu arrow, and then click Edit Web Part.
- In the Web Part tool pane, in the Properties section, in the Search Criteria section, click Change query.
- On the BASICS tab, do one of the following:
- To define your query by using Quick Mode, select options as described in the following table:
-
Quick Mode (default)
|
Select a query |
Select a result source to specify which content should be searched. If you have shared a document library or list as catalog, the catalog result source will be displayed in this drop-down list. By default, this is set to Recently changed items (System). |
||||||||
|
Restrict results by app |
Select an option from the list to restrict results to a specific site, library, list, or URL. By default, this is set to Current site. |
||||||||
|
Restrict by tag |
You can limit results to content that is tagged with a term from a term set. Select one of the following options:
|
- To create your own query by using Keyword Query Language (KQL), click Switch to Advanced Mode. For information about KQL, see Keyword Query Language (KQL) syntax reference. When you configure the query in Advanced Mode, you can also use query variables. Query variables are placeholders for values that change dynamically depending on the context of the page when the page that contains the Content Search Web Part is being displayed. The correct information is inserted dynamically from the context the query is sent to the index. Examples of query variables are {User.Name}, which represents the name of the user who is viewing the page, or {searchBoxQuery}, which represents the query that a user typed in a search box. Select options as described in the following table:
-
Advanced Mode
|
Select a query |
Select a result source to specify which content should be searched. Default result source is Local SharePoint Results (System). |
|
Keyword filter |
You can use keyword filters to add query variables to your query. See Query variables in SharePoint Server 2013 for a list of available query variables. You can select pre-defined query variables from the drop-down list, and then add them to the query by clicking Add keyword filter. |
|
Property filter |
You can use property filters to query the content of managed properties that are set to queryable in the search schema. You can select managed properties from the Property filter drop-down list. Click Add property filter to add the filter to the query. |
|
Query text |
Type your query by using Keyword Query Language (KQL), or use the Keyword filter and Property filter lists to build the query. The keyword query can consist of free-text keywords, property filters, or operators. Use braces to enclose query variables. The query variables will be replaced with an actual value when the query is run. Keyword queries have a maximum length of 2,048 characters. |
-
The REFINERS tab lists the managed properties that are enabled as refiners in the search schema. You can specify that the search results returned in the Content Search Web Part should be limited to one or more values from the refiners. Click a refiner in the list, and then click Apply to add it to the query.
Click Show more if you want to define grouping of results. Under Group results, you can specify that the results should be grouped based on one or more managed properties. This is useful when you are displaying several variants for a given item, and want to group them under a single result.
-
On the SORTING tab, you can specify how search results should be sorted.
This tab is available only if you use Advanced Mode. If you use Quick Mode, you can define sorting options in the result source.
In the Sort by drop-down list, select a managed property from the list of managed properties that are set as sortable in the search schema, and then select Descending or Ascending. For example, to sort by relevance (that is, to use a ranking model) select Rank.
To add more sorting levels, click Add sort level.
If you selected Rank from the Sort by list, you can select which ranking model to use for sorting in the Ranking Model list.
Under Dynamic ordering, you can specify additional ranking by adding rules that will change the order of results when certain conditions apply. Click Add dynamic ordering rule, and then specify conditional rules.
- On the SETTINGS tab, specify the settings that are listed in the following table.
|
Query Rules |
Select whether to use Query Rules or not. |
|
URL Rewriting |
Select if the URL rewrite to the item details page should continue to be relative for each catalog item as defined when you set up the catalog connection. If you select Don’t rewrite URLs, the URLs for catalog items are pointed directly to the library item of the connected catalog. |
|
Loading Behavior |
Select when the search results returned by the Content Search Web Part appear on the web page. The default option is Sync option: Issue query from the server. By using this loading behavior, queries are issued from the server, and the search results are included in the page response that is sent back from SharePoint. If you select Async option: Issue query from the browser, the queries will be issued from the end-users browser after the complete page is received. This option may be considered for secondary content on a page — for example Recommendations or Popular Items. |
|
Priority |
Select the level that best describes the relative importance of content that is displayed by this Web Part in relation to other Search Web Parts. If SharePoint Server 2013 is running under heavy load, the queries will be run according to their priority. |
- On the TEST tab, you can preview the query that is sent by the Content Search Web Part.
|
Query text |
Shows the final query that will be run by the Content Search Web Part. It is based on the original query template where dynamic variables are substituted with current values. Other changes to the query may have to be made as part of query rules. |
Click Show more to display additional information.
|
Query template |
Shows the content of the query template that is applied to the query. |
|
Refined by |
Shows the refiners applied to the query as defined on the REFINERS tab. |
|
Grouped by |
Shows the managed property on which search results should be grouped as defined on the REFINERS tab. |
|
Applied query rules |
Shows which query rules are applied to the query. |
The Query template variables section shows the query variables that will be applied to the query, and the values of the variables that apply to the current page. You can type other values to test the effect they will have on the query. Click the Test Query button to preview the search results.
You can also test how the query works for different user segment terms. Click Add user segment term to add terms to be added to the query. Click the Test query button to preview the search results.
|
Shows the final query that will be run by the Content Search Web Part. It is based on the original query template where dynamic variables are substituted with current values. Other changes to the query may have to be made as part of query rules. |
-
Configure the display templates for the Content Search Web Part
When you connect a publishing site to a catalog, the default control display template for the Content Search Web Part on your category page is List with Paging (named Control_ListWithPaging in the Master Page Gallery).
The default item display template for the Content Search Web Part is Picture on top, 3 lines on bottom (named Item_Picture3Lines in the Master Page Gallery). If you want to use other display templates on your category page, you can change them by changing the settings for the Content Search Web Part.
-
Add a Refinement Web Part to a page
You can add refiners to a page to narrow the items that are shown in a Content Search Web Part, and help users quickly browse to specific content. Refiners are based on managed properties from the search index. To display refiners on a page, you must first enable the managed property that you want to use as a refiner, and then add a Refinement Web Part to the page where you want the refiners to appear. You can configure the Refinement Web Part for two types of refiners: Stand-alone refiners and Refiners for faceted navigation. For more information about the different refiner types, see Plan refiners and faceted navigation in Plan search for SharePoint cross-site publishing sites (SharePoint 2013 Preview).
Before you begin this procedure, verify the following:
- The managed properties that you want to use as refiners are enabled as refinable managed properties described in “Enable a managed property as refiner” in Configure refiners and faceted navigation in SharePoint Server 2013.
- You have done a full crawl of the content source that contains the managed properties that are enabled as refiners as described in Start, pause, resume, or stop crawls in SharePoint 2013 Preview.
- If you are using refiners for faceted navigation, you have configured the refiners as described in “Configure refiners for faceted navigation” in Configure refiners and faceted navigation in SharePoint Server 2013.
To add a Refinement Web Part to a page
- The managed properties that you want to use as refiners are enabled as refinable managed properties described in “Enable a managed property as refiner” in Configure refiners and faceted navigation in SharePoint Server 2013.
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you want to add the Web Part.
- Click the Settings menu, and then click Edit Page.
- In the Web Part Zone where you want to add the Web Part, click Add a Web Part.
- In the Categories list, select Search.
- In the Parts list, select Refinement, and then click Add.
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page that contains the Refinement Web Part that you want to configure.
- Click the Settings menu, and then click Edit Page.
- In the Web Part, click the Refinement Web Part Menu arrow, and then click Edit Web Part.
- You can configure the Web Part for stand-alone refiners or for refiners for faceted navigation by using the following procedures,
- To configure the Web Part for stand-alone refiners:
- In the Web Part tool pane, in the Properties for Search Refinement section, verify that the Choose Refiners in this Web Part is selected.
- Click Choose Refiners…
- On the Refinement configuration page, from the Available refiners section, use the buttons to select which refiners should be added to the term set, and also in which order that they should be displayed. If you have specified an alias for a refinable managed property, this alias is displayed in the Configuration for section.
- In the Configuration for section, set the configuration for how every refiner appears.
Note:
If you have a single language site, you can change the refiner display name in the Display name section. For multilingual sites, you have to change the refiner display language as described in Change the refiner display name.
- To configure the Web Part for refiners for faceted navigation:
- In the Web Part tool pane, in the Properties for Search Refinement section, select the option Use the refinement configuration defined in the Managed Navigation term set.
-
Change the refiner display name
When you add a Refinement Web Part, the name of the managed property that is enabled as a refiner will be used as display name for the refiner. In many cases, the managed property name is not user-friendly — for example, RefinableString00 or ColorOWSTEXT. You can change the display name of the refiner by changing a java script file in the master page gallery.
To change the refiner display name
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- On the Settings menu, click Site Settings.
- On the Site Settings page, in the Web Designer Galleries section, click Master pages and page layouts.
- On the Master Page Gallery page, click Display Templates.
- On the Display Templates page, click Language Files.
- On the Language Files page, click the folder that contains the language that you want to change the refiner display name for.
- Open the CustomStrings.js file.
-
Add one line to the file for each managed property that is enabled as a refiner for which you want to change the display name byusing the following syntax:
“rf_RefinementTitle_ManagedPropertyName”: “Sample Refinement Title for ManagedPropertyName”
For example, you can add the following line to change the display name for the managed property RefinableInt00 to Price:
“rf_RefinementTitle_RefinableInt00”: “Price”.
-
Display refiner counts in a Refinement Web Part
When you add a Refinement Web Part to a page, by default, the Web Part will not show refiner counts — that is, the number of items for each refiner value. For example, if you have enabled the managed property Color as a refiner, the refiner values will only show colors such as Red, Green, and Blue. You can add refiner counts by changing a value in an HTML file so that the refiner values are shown as Red (10), Green (12), and Blue (8).
To add refiner counts to the Refinement Web Part
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- On the Settings menu, click Site Settings.
- On the Site Settings page, in the Web Designer Galleries section, click Master pages and page layouts.
- On the Master Page Gallery page, click Display Templates.
- On the Display Templates page, click Filters.
- Open the Filter_Default.html file.
- Change the value for ShowCounts to true.
-
Configure the display templates for the Refinement Web Part
The display templates for the Refinement Web Part can be found in the Master Page Gallery.
To configure display templates for the Refinement Web Part
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- On the Settings menu, click Site Settings.
- On the Site Settings page, in the Web Designer Galleries section, click Master pages and page layouts.
- On the Master Page Gallery page, click Display Templates.
- On the Display Templates page, click Filters.
You can change the display template that is used by each refiner by selecting a display template from a list in the Display template section on the Refinement configuration page. When you add a Filter display template to the master page gallery, it is added to the list.
-
Add a Taxonomy Refinement Panel Web Part to a page
Before you begin this procedure, verify the following:
- The managed properties that you want to use as refiners are enabled as refinable as described in Enable a managed property as refiner (SharePoint 2013 Preivew).
- You have done a full crawl of the content source that contains the managed properties that are enabled as refiners as described in Start, pause, resume, or stop crawls in SharePoint 2013 Preview.
- If you are using refiners for faceted navigation, you have configured the refiners as described in Configure refiners for faceted navigation (SharePoint 2013 Preview).
To add a Taxonomy Refinement Panel Web Part to a page
- The managed properties that you want to use as refiners are enabled as refinable as described in Enable a managed property as refiner (SharePoint 2013 Preivew).
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you want to add the Web Part.
- Click the Settings menu, and then click Edit Page.
- In the Web Part Zone where you want to add the Web Part, click Add a Web Part.
- In the Categories list, select Search.
- In the Parts, select Taxonomy Refinement Panel, and then click Add.
-
Configure the Taxonomy Refinement Panel Web Part
To configure the Taxonomy Refinement Panel Web Part
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you have the Taxonomy Refinement Panel Web Part that you want to configure.
- On the Settings menu, click Edit Page.
- In the Web Part, click the Taxonomy Refinement Panel Web Part Menu arrow, and then click Edit Web Part.
- In the Web Part tool pane, in the Properties section, in the Query section, on the Refinement Target menu, select the Web Part you want to associate with the Taxonomy Refinement Panel Web Part.
- In the Web Part tool pane, in the Properties section, in the Query section, on the Refiner menu, select the managed property that you have specified for Managed Navigation.
-
Add a Recommended Items Web Part to a page
You can use the Recommended Items Web Part to show content recommendations based how users have previously interacted with the site. For example, you can add this Web Part to a Catalog Item page. If a user views a specific item, this Web Part will display other items that users have previously viewed, such as “Users who viewed this item also viewed these items.“ For more information about recommendations, see Plan usage analytics, usage events and recommendations in Plan search for SharePoint cross-site publishing sites (SharePoint 2013 Preview).
To add a Recommended Items Web Part to a page
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you want to add the Web Part.
- Click the Settings menu, and then click Edit Page.
- In the Web Part Zone where you want to add the Web Part, click Add a Web Part.
- In the Categories list, click Search-Driven Content.
- In the Parts list, click Recommended Items, and then click Add.
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the publishing site collection.
- Browse to the page where you have the Recommended Items Web Part that you want to configure.
- On the Settings menu, click Edit Page.
- In the Web Part, click the Recommended Items Web Part Menu arrow, and then click Edit Web Part.
- In the Web Part tool pane, in the Properties section, in the Search Criteria section, click Change query.
- On the BASICS tab, define your query by selecting options described in the following table.
|
From the drop-down list, select from which value recommendations should be displayed. In a catalog scenario, this will often be A token from a URL. If you select this option, you will also have to select which URL token you want to obtain recommendations for. For example, let’s say that you want to obtain recommendations for items in your catalog. You have a catalog item page where you display your catalog items, and the item number is part of your friendly URL — for example, www.contoso/audio/mp3/4010101. (4010101 represents the item number.) When you want to obtain recommendations for a token from the URL, you should select {URLToken.1} (4010101) from the second drop-down list. |
|
Restrict results by app |
Use this drop-down list to specify a scope for the search results. |
|
Restrict results by content type |
Use this drop-down list to limit the search results to a specific content type. |
|
If there are too few recommended items |
If you don‘t have much usage data — for example, if your site is fairly new, or if the items do not have recommendations to display — this Web Part will not display any search results. In order for the Web Part to display recommendations even though not enough user data has cumulated, you can select the option to Select a query to fill in with additional results. |
-
The REFINERS tab lists the managed properties that are set as refiner-enabled in the search schema. You can specify that the search results returned in the Recommended Items Search Web Part should be limited to one or more values from the refiners. Click a refiner in the list, and then click Apply to add it to the query.
Click Show more if you want to define grouping of results. Under Group results, you can specify that the results should be grouped based on one or more managed properties.
- On the SETTINGS tab, specify the following:
|
Select whether to use Query Rules or not. |
|
URL Rewriting |
Select if the URL rewrite to the item details page should continue to be relative for each catalog item as defined when you set up the catalog connection. If you select Don’t rewrite URLs, the URLs for your catalog items are pointed directly to the library item of the connected catalog. |
|
Loading Behavior |
Select when the search results returned by the Recommended Items Web Part should be displayed on the web page. The default option is Display the page and web party simultaneously. By using this loading behavior, queries are issued from the server, and the search results are included in the page response that is sent back from SharePoint. If you select Display the page and web part independently, the queries will be issued from the end-users browser after the complete page is received. This option may be considered for secondary content on a page — for example, Recommendations or Popular Items |
|
Priority |
Select the level that best describes the relative importance of content that is displayed by this Web Part in relation to other Search Web Parts. If SharePoint Server 2013 is running under heavy load, the queries will be run according to their priority. |
- On the TEST tab, you can preview the query that is sent by the Recommended Items Web Part.
|
Shows the content of the query template that is applied to the query. |
Click Show more to display additional information the query is
|
Shows the refiners applied to the query as defined in the REFINERS tab. |
|
Grouped by |
Shows the managed property on which search results should be grouped as defined in the REFINERS tab. |
|
Applied query rules |
Shows which query rules are applied to the query. |
In the Query template variables section, the selections that you made on the BASIC tab are displayed. In addition, you can type additional values for testing as outlined in the following table. Click the Test query button to preview the search results.
|
Shows the token you selected when specifying for which value recommendations should be displayed. |
|
{Scope}* |
Shows the scope that you selected for the search results. |
|
{ContentTypeID}* |
Shows the content type that you selected for the search results. |
You can also test how the query works for different user segment terms. Click Add user segment term for testing to add terms to be added to the query. Click the Test query button to preview the search results.
|
Shows the final query that will be run by the Recommended Items Web Part. It is based on the original query template where dynamic variables are substituted with current values. Other changes to the query may have be made as part of query rules. |
-
Configure the display templates for the Recommended Items Web Part
The default control display template for the Recommended Items Search Web Part is List (known as Control_List in the Master Page Gallery).
The default item display template for the Recommended Items Web Part is Recommended Items: Picture on top, 3 lines (known as Item_RecommendationsClickLogging in the Master Page Gallery). When a user clicks a link that is displayed in the Recommended Items Web Part, the default display template logs a Recommendations Clicked usage event.
-
Configure refiners and faceted navigation in SharePoint Server 2013
Published: October 2, 2012
Summary: Learn how to map a crawled property to a refinable managed property, enable a managed property as a refiner and configure refiners for faceted navigation.
Applies to: SharePoint Server 2013
You can add refiners to a page to help users quickly browse to specific content. Refiners are based on managed properties from the search index. To use managed properties as refiners, the managed properties must be enabled as refiners.
Faceted navigation is the process of browsing for content by filtering on refiners that are tied to category pages. Faceted navigation allows you to specify different refiners for category pages, even when the underlying page displaying the categories is the same. For information about category pages, see “Category pages and catalog item pages” in Overview of cross-site publishing in SharePoint Server 2013 Preview.
Important:
You can apply faceted navigation only to publishing sites that use managed navigation. You configure the refiners that are used in faceted navigation on the term set on the authoring site collection.
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
Review the information in “Plan refiners and faceted navigation” in Plan search for SharePoint cross-site publishing sites (SharePoint 2013 Preview).
- Plan browser support
-
Enable a managed property as refiner
To use a managed property as refiner, the managed property must be enabled as refiner. Search service application administrators can do this in Central Administration as described in Enable a managed property as a refiner in SharePoint Central Administration later in this article.
Site collection administrators can configure refiners because the search schema has many managed properties that are enabled as refiners by default. These managed properties are listed in the following table. Before site collection administrators can use these managed properties as refiners on their web pages, they must map the appropriate crawled property to the managed property that is enabled as a refiner. To make it easier to work with these properties when doing additional refiner configuration in Term Store Management, you can specify a user-friendly alias name for the managed property.
-
Managed properties that are enabled as refiners by default
Managed property name
Data type for mapping
RefinableDate00 – RefinableDate19
Values contain dates
RefinableDecimal00 – RefinableDecimal09
Values contain numbers with maximum three decimals
RefinableDouble00 – RefinableDouble09
Values contain numbers with more than three decimals
RefinableInt00 – RefinableInt49
Values are whole numbers
RefinableString00 – RefinableString99
Values are strings
Note:
We recommend that you use only the managed properties that are enabled as refiners by default when you perform the procedure in the following section.
To map a crawled property to a refinable managed property
-
- Verify that the user account that performs this procedure has the following credentials:
- The user account that performs this procedure is a site collection administrator on the publishing site collection.
- On the publishing site collection, on the Settings menu, click Site settings.
- On the Site Settings page, in the Site Collection Administration section, click Search Schema.
- On the Managed Properties page, in the Managed property filter box, type the name of a refinable managed property — for example, RefinableString00 — and then click the arrow.
- In the Property Name column, click the refinable managed property that you want to edit.
- To specify an alias of the refinable managed property to use when you configure refiners for faceted navigation, on the Edit Managed Property page, type a user-friendly name in the Alias box.
- In the Mappings to crawled properties section, click Add a Mapping.
- In the Crawled property selection dialog box, find the crawled property that you want to map to the refinable managed property in the list, or search for it by typing the name of the crawled property in the box, and then clicking Find.
Important:
When you search for a crawled property, you may find two crawled properties that represent the same content. For example, a site column of type text named Color will during crawl discover two crawled properties: ows_Color and ows_q_TEXT_Color. Crawled properties that begin with either ows_r<four letter code>, ows_q<four letter code> or ows_taxId are automatically created crawled properties. When you select a crawled property to map to a refinable managed property, make sure that you don’t map the automatically created crawled property. You should always map the crawled property that begins with ows_.
For information about automatically created crawled properties, see About automatically created managed properties (SharePoint 2013 Preview).
- Click OK.
- On the Edit Managed Property page, click OK.
Note:
To configure refiners in Web Parts or in Term Store Management, you must start a full crawl of the content source that contains the refinable managed properties. For more information, see Start, pause, resume, or stop crawls in SharePoint 2013 Preview.
Important:
All automatically created managed properties use the text data type. Therefore, you should only enable an automatically created managed property as a refiner if the site column used to create the managed property also uses the text data type. For example, if the site column uses an integer or date data type, you must create a new managed property, map the crawled property value to this new managed property, and then enable it as a refiner.
When you select a crawled property to map to a managed property, make sure that you don‘t map the automatically created crawled property. The name of the automatically created crawled property starts with either ows_r<four letter code>_, ows_q<four letter code>_, or ows_taxId_. The name of the crawled property that you should use in the mapping starts with ows_.
For information about how to create a new managed property, see To add a managed property. For more information about automatically created crawled properties, see About automatically created managed properties (SharePoint 2013 Preview).
To enable a managed property as a refiner
- Verify that the user account that performs this procedure is an administrator of the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- On the Manage Service Applications page, click Search Service Application.
- Click the Search service application.
- On the Managed Properties page, in the Managed property filter box, type the name of the managed property that you want to enable as refiner, and then click the arrow.
- In the Property Name column, click the managed property that you want to edit.
- On the Edit Managed Property page, in the Refinable section, select either Yes – active or Yes – latent. If you select Yes – latent, you can switch the refiner to active later without having to do a full crawl.
- Click OK.
Note:
To configure refiners in Web Parts or in Term Store Management, a full crawl of the content source that contains the refinable managed properties must be completed. Administrators of the Search service application can complete a full crawl as described in Start, pause, resume, or stop crawls in SharePoint 2013 Preview. Site collection administrators can initiate a full crawl by specifying that the catalog that contains the refinable managed properties should be reindexed during the next scheduled crawl.
-
Configure refiners for faceted navigation
Before you start the procedures in this section, verify the following:
- On the authoring site, a library or list is shared as a catalog, as described in “Share a library or list as a catalog” in Configure cross-site publishing in SharePoint Server 2013.
- The required managed properties are enabled as refiners, as described in Enable a managed property as refiner.
-
A full crawl was completed for the content source that contains the refinable managed properties, as described in Start, pause, resume, or stop crawls in SharePoint 2013 Preview.
To configure refiners for faceted navigation, you must first enable the relevant term set for faceted navigation. This procedure is performed on the authoring site collection.
To enable a term set for faceted navigation
- On the authoring site, a library or list is shared as a catalog, as described in “Share a library or list as a catalog” in Configure cross-site publishing in SharePoint Server 2013.
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the authoring site collection.
- On the authoring site collection, on the Settings menu, click Site settings.
- On the Site Settings page, in the Site Administration section, click Term store management.
- In the TAXONOMY TERM STORE section, click to select the term set that you want to enable for faceted navigation.
- Click the INTENDED USE tab, and then select Use this Term Set for Faceted Navigation.
- Click Save.
When configuring refiners for faceted navigation, you can add refiners to all terms in a term set or to specific terms in a term set. This procedure is performed on the authoring site collection.
To add refiners to all terms in a term set
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the authoring site collection.
- On the authoring site collection, on the Settings menu, click Site settings.
- On the Site Settings page, in the Site Administration section, click Term store management.
- In the TAXONOMY TERM STORE section, click the term set that you have enabled for faceted navigation.
- Click the FACETED NAVIGATION tab, and then click Customize refiners….
- On the Refinement Configuration page, in the Available refiners section, use the buttons to select which refiners should be added to the term set, and also to specify the order in which you want the refiners to appear. If you have specified an alias for a refinable managed property, this alias is displayed in the Configuration section.
- In the Configuration for section, specify how you want each refiner to appear.
- Click OK to close the Refinement Configuration page, and then click Save.
To add refiners to specific terms in a term set
- Verify that the user account that performs this procedure is a member of the Designers SharePoint group on the authoring site collection.
- On the authoring site collection, on the Settings menu, click Site settings.
- On the Site Settings page, in the Site Administration section, click Term store management.
- In the TAXONOMY TERM STORE section, click the term set that you have enabled for faceted navigation, and then click the term to which you want to add term-specific refiners.
- Click the FACETED NAVIGATION tab, and then click Stop inheriting….
- Click FACETED NAVIGATION tab, and then click Customize refiners….
- On the Refinement Configuration page, in the Available refiners section, use the buttons to select which refiners should be added to the term set, and also to specify the order in which you want the refiners to appear. If you have specified an alias for a refinable managed property, this alias is displayed in the Configuration section.
- In the Configuration for section, specify how you want each refiner to appear.
- Click OK to close the Refinement Configuration page, and then click Save.
For refiners that contain numeric values, you can present the numeric values within different intervals. For example, if you want end-users to be able to refine based on price, it would be useful to specify different price intervals instead of showing all available prices as separate refiners. This procedure is performed in your authoring site collection.
To set intervals for refiner values
- Add refiners to a term set as described in Add refiners to a term set in this topic.
- On the Refinement Configuration page, in the Selected refiners section, click the refiner that you want to set intervals for.
- In the Configuration for section, for Intervals, select Custom, and then type the intervals in the Thresholds box.
- Click OK to close the Refinement Configuration page, and then click Save.
-
Additional steps
To show refiners on a page, you must add a Refinement Panel Web Part to the page where you want the refiners to appear. For more information, see Configure Search Web Parts in SharePoint Server 2013.
-
Configure result sources for web content management in SharePoint Server 2013
Published: October 16, 2012
Summary: Learn how to create and manage result sources for SharePoint Search service applications, and for SharePoint sites and site collections.
Applies to:
Result sources limit searches to certain content or to a subset of search results. SharePoint Server 2013 provides 16 pre-defined result sources. The pre-configured default result source is Local SharePoint Results. You can specify a different result source as the default. In addition to the pre-configured result sources, SharePoint Server 2013 automatically creates a result source when you connect a publishing site to a catalog, and adds it to the result sources in the publishing site. This result source limits search results to the URL of the catalog. For more information about result sources, see “Plan result sources and query rules” in Plan search for cross-site publishing sites in SharePoint Server 2013 Preview.
In this article:
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support
- Accessibility for SharePoint 2013
- Accessibility features in SharePoint 2013 Products
- Keyboard shortcuts
- Touch
We recommend that you set up the publishing site, integrate a catalog, and configure category and catalog item pages before you begin to create result sources. This is because you can then more easily test and verify how the different result sources apply to the different Search Web Parts that you have on the site.
- Plan browser support
-
Create a result source
You can create a result source for a Search service application, a site collection, or a site. The following table shows the permissions that are required to create a result source at each level, and where the result source can be used.
-
Levels and permissions for result sources
When you create a result source at this level
You must have this permission
The result source can be used in
Search service application
Search service application administrator
All site collections in web applications that consume the Search service application
Site collection
Site collection administrator
All sites in the site collection
Site
Site owner
The site
To create a result source
-
- Depending on the level at which you want to create the result source, do one of the following:
- To create a result source for a Search service application:
- Verify that the user account that performs this procedure is an administrator on the Search service application.
- In Central Administration, in the Application Management section, click Manage service application.
- Click the Search service application for which you want to create a result source.
- On the Search Administration page for the Search service application, on the Quick Launch, in the Queries and Results section, click Result Sources.
- To create a result source for a site collection:
- Verify that the user account that performs this procedure is a site collection administrator on the publishing site collection.
- On the publishing site collection, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Collection Administration section, click Search Result Sources.
- To create a result source for a site:
- Verify that the user account that performs this procedure is a member of the Owners group on the publishing site.
- On the publishing site, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Search section, click Result Sources.
- On the Manage Result Sources page, click New Result Source.
-
On the Add Result Source page, in the General Information section, do the following:
- In the Name box, type a name for the result source.
- In the Description box, type a description of the result source.
- In the Name box, type a name for the result source.
- In the Protocol section, select one of the following protocols for retrieving search results:
- Local SharePoint, the default protocol, provides results from the search index for this Search service application.
- Remote SharePoint provides results from the index of a search service in another farm.
- OpenSearch provides results from a search engine that uses the OpenSearch 1.0/1.1 protocol.
-
Exchange provides results from Microsoft Exchange Server. Click Use AutoDiscover to have the search system find an Exchange Server endpoint automatically, or type the URL of the Exchange web service to retrieve results from — for example, https://contoso.com/ews/exchange.asmx.
Note:
Note: The Exchange Web Services Managed API must be installed on the computer on which the search service is running. For more information, see Optional software in Hardware and software requirements for SharePoint 2013.
- In the Type section, select SharePoint Search Results to search the whole index, or People Search Results to enable query processing that is specific to people search.
- In the Query Transform field, do one of the following:
- Leave the default query transform (searchTerms) as is. In this case, the query will be unchanged since the previous transform.
- Type a different query transform in the text box.
- Use the Query Builder to configure a query transform by doing the following:
- Click Launch Query Builder.
- In the Build Your Query dialog box, optionally build the query by specifying filters, sorting, and testing on the tabs as shown in the following tables.
-
On the BASICS tab
|
Keyword filter |
You can use keyword filters to add pre-defined query variables to the query transform. You can select pre-defined query variables from the drop-down list, and then add them to the query by clicking Add keyword filter. For an overview of query variables, see Query variables in SharePoint Server 2013. |
|
Property filter |
You can use property filters to query the content of managed properties that are set to queryable in the search schema. You can select managed properties from the Property filter drop-down list. Click Add property filter to add the filter to the query. |
-
On the SORTING tab
|
Sort results |
In the Sort by menu, you can select a managed property from the list of managed properties that are set as sortable in the search schema, and then select Descending or Ascending. To sort by relevance, that is, to use a ranking model, select Rank. You can click Add sort level to specify a property for a secondary level of sorting for search results. |
|
Ranking Model |
If you selected Rank from the Sort by list, you can select the ranking model to use for sorting. |
|
Dynamic ordering |
You can click Add dynamic ordering rule to specify additional ranking by adding rules that change the order of results within the result block when certain conditions are satisfied. |
-
On the TEST tab
|
Query text |
You can view the final query text, which is based on the original query template, the applicable query rules, and the variable values. |
|
Click Show more to display the options in the following rows of this table. |
|
|
Query template |
You can view the query as it is defined in the BASICS tab or in the text box in the Query transform section on the Add Result Source page. |
|
Query template variables |
You can test the query template by specifying values for the query variables. |
- On the Add Result Source page, in the Credentials Information section, select the authentication type that you want for users to connect to the result source.
-
Set a result source as default
You can set any result source as the default result source. Specifying a result source as default can make it easier to edit the query in Search Web Parts. For example, when you add a Content Search Web Part to a page, the Web Part automatically uses the default result source. For more information, see Configure Search Web Parts in SharePoint Server 2013.
To set a result source as default
- Perform the appropriate procedures in the following list depending on the level at which the result source was configured.
- If the result source was created at the Search service application level, do the following:
- Verify that the user account that performs this procedure is an administrator for the Search service application.
- In Central Administration, in the Application Management section, click Manage service applications.
- Click the Search service application for which you want to set the result source as default.
- On the Search Administration page, in the Queries and Results section, click Result Sources.
- If the result source is at the site collection level, do the following:
- Verify that the user account that performs this procedure is a site collection administrator on the publishing site collection.
- On the publishing site collection, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Site Collection Administration section, click Search Result Sources.
- If the result source is at the site level, do the following:
- Verify that the user account that performs this procedure is a member of the Owners group on the publishing site.
- On the publishing site, on the Settings menu, click Site Settings.
- On the Site Settings page, in the Search section, click Result Sources.
- On the Manage Result Sources page, point to the result source that you want to set as default, click the arrow that appears, and then click Set as Default.
-
Configure recommendations and usage event types in SharePoint Server 2013
Published: October 16, 2011
Applies to:
Usage events enable you to track how users interact with items on your site. Items can be documents, sites, or catalog items. When a user interacts with an item on your site, SharePoint Server 2013 generates a usage event for this action. For example, if you want to monitor how often a catalog item is viewed from a mobile phone, you can track this activity.
This article describes how to create custom usage event types, and how to add code to record custom usage events so that they can be processed by the analytics processing component.
You can use the data that is generated by usage events to show recommendations or popular items on your site. This article also explains how to influence how recommendations are shown by changing the level of importance for a specific usage event type. For more information, see “Plan usage analytics, usage events and recommendations” in Plan search for cross-site publishing sites in SharePoint Server 2013 Preview.
You can view the statistics for all usage event types in Popularity Trends and Most Popular Items reports. For more information, see View usage reports.
In this article:
-
Before you begin
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Create a custom usage event type
There are three default usage event types in SharePoint 2013. You can create up to twelve custom usage event types by using Windows PowerShell.
To create a custom usage event type
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To get a site at the root site collection level:
$Site = Get-SPSite “http://localhost”# To get a site below the root site collection level:
$Site = Get-SPSite “http://localhost/sites/<SiteName>”# To create a custom usage event type:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$EventGuid = [Guid]::NewGuid()
$EventName = “<EventTypeName>”
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$newEventType = $tenantConfig.RegisterEventType($EventGuid, $EventName, “”)
$tenantConfig.Update($SSP)
Where:
- <SiteName> is the name of the site for which you want to create a custom usage event.
-
<EventTypeName> is the name of the new custom usage event type that you want to create — for example, BuyEventType.
This procedure creates a random GUID for the usage event type. Use this GUID when you add code to record the custom usage event, as described in Record a custom usage event.
Important:
It can take up to three hours for a custom usage event type to become available in the system. However, to speed up the process, you can alternatively restart the SharePoint Timer Service.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Record a custom usage event
After you have created a custom usage event type, as described in Create a custom usage event type, you have to add code to the place where the event occurs — for example, when a page loads, or when a user clicks a link or a button. This data is then sent to the analytics processing component, where it is recorded and processed.
If you are using cross-site publishing, where you show catalog content on a publishing site, you must record the usage event on the URL of the indexed item, and override some site settings. For example, if you have a catalog in an authoring site that you have published on a publishing site, when a user interacts with a catalog item on the publishing site, this usage event must be recorded on the item in the authoring site. Furthermore, the code that you add to record the usage event must override the SiteId and the WebId of the publishing site, and be replaced with the SiteId and the WebId of the authoring site.
To add code to record a custom usage event
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view GUIDs for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft
-
In an HTML editor, open the file where the custom usage event should be logged — for example, a display template for a Content Search Web Part, and add the following code:
window.Log<CustomUsageEventType>ToEventStore = function(url)
{
ExecuteOrDelayUntilScriptLoaded(function()
{
var spClientContext = SP.ClientContext.get_current();
var eventGuid = new SP.Guid(“<GUID>”);
SP.Analytics.AnalyticsUsageEntry.logAnalyticsAppEvent(spClientContext, eventGuid, url);
spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert(“Failed to log event for item: ” + document.URL + ” due to: ” + e.get_message()) }));
}, “SP.js”);
}Where:
- <CustomUsageEventType> is the name of the custom event — for example, BuyEventType.
- <GUID> is the numeric ID of the usage event type — for example, 4e605543-63cf-4b5f-aab6-99a10b8fb257.
-
In an HTML editor, open the file that refers to the custom usage event, and add the following code:
# The example below shows how a custom usage event type is referred to when a button is clicked:
<button onclick=”Log<CustomUsageEventType>ToEventStore(‘<URL>’)”></button>
Where:
- <CustomUsageEventType> is the name of the custom event type.
- <URL> is the full URL of the item to which the usage event should be logged — for example, http://contoso.com/faq.
To add code to record a custom usage event and override site settings
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view GUIDs for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft
-
In an HTML editor, open the file where the custom usage event should be logged — for example, a display template for a Content Search Web Part. The following example shows how to override the current SiteId, WebId and UserId.
window.Log<CustomUsageEventType>ToEventStore = function(url, siteIdGuid, webIdGuid, spUser)
{
ExecuteOrDelayUntilScriptLoaded(function()
{
var spClientContext = SP.ClientContext.get_current();
var eventGuid = new SP.Guid(“<GUID>”);
SP.Analytics.AnalyticsUsageEntry.logAnalyticsAppEvent2(spClientContext, eventGuid, url, webIdGuid, siteIdGuid, spUser);
spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert(“Failed to log event for item: ” + document.URL + ” due to: ” + e.get_message()) }));
}, “SP.js”);
}
Where:
- <CustomUsageEventType> is the name of the custom event type — for example, BuyEventType.
- <GUID> is the numeric ID of the usage event type — for example, 4e605543-63cf-4b5f-aab6-99a10b8fb257.
-
In an HTML editor, open the file that refers to the custom usage event type, and add the following code:
# The example below shows how a custom usage event type is referred to when the “Buy!” button is clicked:
<button onclick=”Log<CustomUsageEventType>ToEventStore(‘<URL>’, new SP.Guid(‘{<SiteId GUID>}’), new SP.Guid(‘{<WebId guid}>’), ‘<UserName>’)”>Buy!</button>
Where:
- <CustomUsageEventType> is the name of the custom event type — for example, BuyEventType.
- <URL> is the URL found in the managed property OriginalPath.
- <SiteId GUID> is the GUID in the managed property SiteID.
- <WebId GUID> is the GUID in the managed property WebId.
- <UserName> can — for example, be a cookie ID that is used to identify users on a site that has anonymous users.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Record a default usage event
If you want to add code that refers to a default usage event type — for example, views, you have to add code to the place where the event occurs.
If you are using cross-site publishing, which shows catalog content on a publishing site, you must record the usage event on the URL of the indexed item, and override some site settings. For example, if you have a catalog in an authoring site that you have published on a publishing site, when a user interacts with a catalog item on the publishing site, this usage event must be recorded on the item in the authoring site. Furthermore, the code that you add to record the usage event must override the SiteId and WebId of the publishing site, and be replaced with the SiteId and WebId of the authoring site.
To add code to record a default usage event
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft
-
In an HTML editor, open the file where the custom usage event should be logged — for example, a display template for a Content Search Web Part, and add the following code:
window.Log<DefaultUsageEventType>ToEventStore = function(url)
{
ExecuteOrDelayUntilScriptLoaded(function()
{
var spClientContext = SP.ClientContext.get_current();
SP.Analytics.AnalyticsUsageEntry.logAnalyticsEvent(spClientContext, <EventTypeId>, url);
spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert(“Failed to log event for item: ” + document.URL + ” due to: ” + e.get_message()) }));
}, “SP.js”);
}
Where:
- <DefaultUsageEventType> is the name of the default usage event type — for example, Views.
- <EventTypeId> is the numeric ID of the usage event type — for example, 1.
-
In an HTML editor, open the file that refers to the default usage event, and add the following code:
# The example below shows how a default usage event type is referred to on a page load:
<body onload=“Log<DefaultUsageEventType>ToEventStore(‘<URL>’)“>
Where:
- <DefaultUsageEventType> is the name of the default usage event type — for example, Views.
- <URL> is the full URL of the item to which the usage event should be logged, — for example, http://contoso.com/careers
- Save the file.
To add code to record a default usage event and override site settings
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft
-
In an HTML editor, open the file where the custom usage event should be logged — for example, a display template for a Content Search Web Part. The example below shows how to override the current SiteId, the WebId and the UserId.
window.Log<DefaultUsageEventType>ToEventStore = function(url, siteIdGuid, webIdGuid, spUser)
{
ExecuteOrDelayUntilScriptLoaded(function()
{
var spClientContext = SP.ClientContext.get_current();
SP.Analytics.AnalyticsUsageEntry.logAnalyticsEvent(spClientContext, <EventTypeId>, url, webIdGuid, siteIdGuid, spUser);
spClientContext.executeQueryAsync(null, Function.createDelegate(this, function(sender, e){ alert(“Failed to log event for item: ” + document.URL + ” due to: ” + e.get_message()) }));
}, “SP.js”);
}
Where:
- <DefaultUsageEventType> is the name of the default event type — for example, Views.
- <EventTypeId> is the numeric ID of the usage event type — for example, 1.
-
In an HTML editor, open the file that refers to the default usage event type, and add the following code:
# The example below shows how a default usage event type is referred to on a page load:
<body onload=“Log<DefaultUsageEventType>ToEventStore(‘<URL>’, new SP.Guid(‘{<SiteId GUID>}’), new SP.Guid(‘{<WebId GUID>}’), ‘<UserName>’)“>
Where:
- <DefaultUsageEventType> is the name of the default event type — for example, Views.
- <URL> is the URL in the managed property OriginalPath.
- <SiteId GUID> is the GUID in the managed property SiteID.
- <WebId GUID> is the GUID in the managed property WebId.
- <UserName><UserName> can — for example, be a cookie ID that is used to identify users on a site that has anonymous users
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Change the level of importance of a usage event type
The usage event type property, RecommendationWeight, is a numeric value that shows the level of importance of a usage event type compared to other usage event types that are used in the recommendations calculation. The default Views usage event type has a preconfigured RecommendationWeight value of 1. The other default usage event types, Recommendations displayed, and Recommendations clicked, and all custom usage event types, have a RecommendationWeight value of 0. To increase the importance of a usage event type in the recommendations calculation, change the value of the RecommendationWeight parameter. The highest level of importance available is 10.
To change the level of importance of a usage event type
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft# To get a usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }# To change the importance level of a usage event type:
$event.RecommendationWeight = <RecommendationWeightNumber>
$tenantConfig.Update($SSP)# To verify the changed importance level for the usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }
$event
Where:
- <EventTypeId> is the numeric ID of the usage event type for which you want to change the weight — for example, 256.
- <RecommendationWeightNumber> is the level of importance that you want to apply to the user event type — for example, 4.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Change the Recent time period for a usage event type
The usage event type property RecentPopularityTimeframe is a numeric value that defines the Recent time period in the Most Popular Items report. The Most Popular Items report shows the most popular items per usage event type for all items in a library or list — for example, the most viewed items in a library or list. The report can be sorted by the time periods Recent or Ever. By default, the Recent time period is set to the last 14 days for each usage event. You can change this to a time period between one and 14 days.
To change the Recent time period for a usage event type
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft# To get a usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }# To change the Recent time span for a usage event type:
$event.RecentPopularityTimeFrame = <TimeFrame>
$tenantConfig.Update($SSP)# To verify the changed Recent time frame for the usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }
$event
Where:
- <EventTypeId> is the numeric ID of the usage event type for which you want to change the Recent time frame — for example, 256.
-
<TimeFrame> is the new Recent time frame that you want to apply to the user event type — for example, 7.
Note:
The system updates any changes to the Recent time period only after the Usage Analytics Timer Job has run.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Enable and disable the logging of usage events of anonymous users
Users that are browsing the contents of a site without being connected to an account are known as anonymous users. Only the Views event type is enabled for the logging of anonymous users. By default, the logging of custom usage events is disabled for anonymous users.
To enable the logging of usage events of anonymous users
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft# To get a usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }# To enable the logging of anonymous users:
$event.Options = [Microsoft.Office.Server.Search.Analytics.EventOptions]::AllowAnonymousWrite
$tenantConfig.Update($SSP)# To verify that the logging of anonymous users has been enabled, i.e. that the Options property is set to AllowAnonymousWrite:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }
$event
Where:
- <EventTypeId> is the numeric ID of the usage event type that you want to enable for the logging of anonymous users — for example, 256.
To disable the logging of usage events of anonymous users
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
# To view EventTypeId for all usage event types:
$SSP = Get-SPEnterpriseSearchServiceApplicationProxy
$SSP.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft# To get a usage event type:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Guid]::Empty)
$event = $tenantConfig.EventTypeDefinitions | where-object { $_.EventTypeId -eq <EventTypeId> }# To disable the logging of anonymous users:
$event.Options = [Microsoft.Office.Server.Search.Analytics.EventOptions]::None
$tenantConfig.Update($SSP)# To verify that logging of anonymous users has been disabled, i.e. that the Options property is set to None:
$tenantConfig = $SSP.GetAnalyticsTenantConfiguration([Gui
Where:
-
<EventTypeId> is the numeric ID of the usage event type that you want to disable for the logging of anonymous users — for example, 256.
Note:
For the default usage event type Views, you cannot disable the logging of anonymous users.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Configure workflow in SharePoint Server 2013
Published: September 11, 2012
Summary: Learn how to install and configure the SharePoint 2013 Workflow platform in SharePoint Server 2013.
Applies to: SharePoint Server 2013
This section describes how to configure workflow in SharePoint Server 2013.
In this section:
- Installing and configuring workflow for SharePoint Server 2013 This article describes how to install and configure the SharePoint 2013 Workflow platform to work with SharePoint Server.
- Installing Workflow Manager certificates in SharePoint Server 2013 Secure Socket Layer (SSL) certificates are used to provide encrypted communication between SharePoint Server 2013 and Workflow Manager. This article describes how to install the SSL certificates.
- Installing and configuring workflow for SharePoint Server 2013 This article describes how to install and configure the SharePoint 2013 Workflow platform to work with SharePoint Server.
-
Installing and configuring workflow for SharePoint Server 2013
Updated: October 16, 2012
Summary: Learn how to install and configure workflow in SharePoint Server 2013.
Applies to: SharePoint Server 2013
This article contains the information and procedures required to configure workflow in SharePoint Server 2013.
In this article:
- Overview
- Before you begin
- Install and configure SharePoint Server 2013
- Install and configure Workflow Manager
- Configure Workflow Manager to work with the SharePoint Server 2013 farm
- Validate the installation
- Troubleshooting
- Related topics
Important:
The steps in this article apply to SharePoint Server 2013. The SharePoint 2013 Workflow platform is not supported in SharePoint Foundation 2013.
- Overview
-
Overview
A new option exists when you build a workflow for SharePoint Server 2013. This option is called Platform Type. The figure shows the Platform Type option when you are creating a new workflow by using SharePoint Designer 2013.
Figure: SharePoint 2013 includes three workflow platform options.
The only platform available when you first install SharePoint Server 2013 is the SharePoint 2010 Workflow platform. The SharePoint 2013 Workflow platform and the Project Server platform require additional steps. The three workflow platforms are outlined in the following table.
-
Workflow Platform types available in SharePoint Server 2013
Platform Type
Platform Framework
Requirements
SharePoint 2010 Workflow
Windows Workflow Foundation 3
Installs automatically with SharePoint 2013 Products.
SharePoint 2013 Workflow
Windows Workflow Foundation 4
Requires SharePoint Server 2013 and Workflow Manager.
Note:
Workflow Manager must be downloaded and installed separately from SharePoint Server 2013. It does not install automatically when you install SharePoint Server 2013.
SharePoint 2013 Workflow – Project Server
Windows Workflow Foundation 4
Requires SharePoint Server 2013, Workflow Manager, and Project Server 2013.
To learn more about workflow development with SharePoint Designer 2013 and other aspects of workflow, see the Workflow in SharePoint 2013 Resource Center.
-
-
Before you begin
Before you begin installation, make sure that you have met all hardware and software requirements for both SharePoint Server 2013 and Workflow Manager. For more information, see Hardware and software requirements (SharePoint 2013 Preview) and Planning your Workflow Manager Deployment.
Important:
The steps in this article apply to SharePoint Server 2013. The SharePoint 2013 Workflow platform is not supported in SharePoint Foundation 2013.
-
Install and configure SharePoint Server 2013
You must install and configure SharePoint Server 2013. To do so, see Install and deploy SharePoint 2013.
Note:
The SharePoint 2010 Workflow platform installs automatically when you install SharePoint Server 2013. The SharePoint 2013 Workflow platform requires Workflow Manager and must be installed separately and then configured to work with your SharePoint Server 2013 farm.
-
Install and configure Workflow Manager
You must install and configure Workflow Manager. To do so, see Installing and Configuring Workflow.
-
Configure Workflow Manager to work with the SharePoint Server 2013 farm
You must consider the following two key factors before configuring Workflow Manager to work with SharePoint Server 2013.
- Is Workflow Manager installed on a server that is part of the SharePoint farm?
- Will communication between Workflow Manager and SharePoint Server 2013 use HTTP or HTTPS?
These factors translate into four scenarios. Each scenario configures a SharePoint Server 2013 farm to communicate and function with the Workflow Manager farm. Follow the scenario that matches your circumstance.
1: Workflow Manager is installed on a server that is part of the SharePoint 2013 farm. Communication takes place by using HTTP.
2: Workflow Manager is installed on a server that is part of the SharePoint 2013 farm. Communication takes place by using HTTPS.
3: Workflow Manager is installed on a server that is NOT part of the SharePoint 2013 farm. Communication takes place by using HTTP.
4: Workflow Manager is installed on a server that is NOT part of the SharePoint 2013 farm. Communication takes place by using HTTPS.
Note:
For security reasons, we recommend HTTPS for a production environment.
To configure Workflow Manager on a server that is part of the SharePoint 2013 farm and on which communication takes place by using HTTP
- Is Workflow Manager installed on a server that is part of the SharePoint farm?
- Log on to the computer in the SharePoint Server 2013 farm where Workflow Manager was installed.
- Open the SharePoint Management Shell as an administrator. This is accomplished by right-clicking the SharePoint 2013 Management Shell and choosing Run as administrator.
-
Run the Register-SPWorkflowService cmdlet.
Example:
Register-SPWorkflowService –SPSite “http://myserver/mysitecollection” –WorkflowHostUri “http://workflow.example.com:12291” –AllowOAuthHttp
To configure Workflow Manager on a server that is part of the SharePoint 2013 farm and on which communication takes place by using HTTPS
-
Determine if you need to install Workflow Manager certificates in SharePoint.
Under some circumstances, you have to obtain and install Workflow Manager certificates. If your installation requires that you obtain and install these certificates, you must complete that step before continuing. To learn whether you need to install certificates, and for instructions, see Installing Workflow Manager certificates in SharePoint Server 2013.
- Log into the computer in the SharePoint Server 2013 farm where Workflow Manager was installed.
- Open the SharePoint Management Shell as an administrator. This is accomplished by right-clicking the SharePoint 2013 Management Shell and choosing Run as administrator.
-
Run the Register-SPWorkflowService cmdlet.
Example:
Register-SPWorkflowService –SPSite “https://myserver/mysitecollection” –WorkflowHostUri “https://workflow.example.com:12290”
To configure Workflow Manager on a server that is NOT part of the SharePoint 2013 farm and on which communication takes place by using HTTP
- Log on to each Web Front End (WFE) server in the SharePoint Server 2013 farm.
-
Install the Workflow Manager Client on each WFE server in the SharePoint farm.
Before you can run the workflow pairing cmdlet, you must install Workflow Manager Client on each of the WFE servers in the SharePoint farm.
You can download and install the Workflow Manager Client here: http://go.microsoft.com/fwlink/p/?LinkID=268376
- Open the SharePoint Management Shell as an administrator. This is accomplished by right-clicking the SharePoint 2013 Management Shell command and choosing Run as administrator.
-
Run the Register-SPWorkflowService cmdlet. The cmdlet should be run only once and can be run from any of the WFE servers in the SharePoint farm. Example:
Register-SPWorkflowService –SPSite “http://myserver/mysitecollection” –WorkflowHostUri “http://workflow.example.com:12291” –AllowOAuthHttp
Important:
You must install the Workflow Manager Client on each Web Front End (WFE) server in the SharePoint farm before you run the pairing cmdlet.
To configure Workflow Manager on a server that is NOT part of the SharePoint 2013 farm and on which communication takes place by using HTTPS
-
Determine whether you need to install Workflow Manager certificates in SharePoint 2013.
Under some circumstances, you have to obtain and install Workflow Manager certificates. If your installation requires that you obtain and install these certificates, you must complete that step before continuing. To learn whether you need to install certificates, and for instructions, see Installing Workflow Manager certificates in SharePoint Server 2013.
- Log on to each Web Front End (WFE) server in the SharePoint Server 2013 farm.
-
Install the Workflow Manager Client on each WFE server in the SharePoint farm.
Before you can run the workflow pairing cmdlet, you must install Workflow Manager Client on each of the WFE servers in the SharePoint farm.
You can download and install the Workflow Manager Client here: http://go.microsoft.com/fwlink/p/?LinkID=268376
- Open the SharePoint Management Shell as an administrator. This is accomplished by right-clicking the SharePoint 2013 Management Shell command and choosing Run as administrator.
-
Run the Register-SPWorkflowService cmdlet. Example:
Register-SPWorkflowService –SPSite “https://myserver/mysitecollection” –WorkflowHostUri “https://workflow.example.com:12290”
Important:
You must install the Workflow Manager Client on each Web Front End (WFE) server in the SharePoint farm before you run the pairing cmdlet.
-
Validate the installation
Use these steps to validate that you have successfully installed and configured the required components.
To validate the installation
- Add a user to your SharePoint site, and grant the user Site Designer permissions.
- Install SharePoint Designer 2013 and create a workflow based on the SharePoint 2013 Workflow platform. For more information, see Creating a workflow by using SharePoint Designer 2013 and the SharePoint 2013 Workflow platform.
- Run this workflow from the SharePoint user interface.
-
Troubleshooting
For security reasons, the Setup account cannot be used to create a workflow based on the SharePoint 2013 Workflow platform. If you try to create a workflow based on the SharePoint 2013 Workflow platform by using SharePoint Designer 2013, you receive a warning that the list of workflow actions do not exist, and the workflow is not created.
The user who deploys and runs a workflow must be added to the User Profile service. Check the User Profile service application page in Central Administration to confirm that the user you are using to validate workflow installation is in the User Profile service.
You can determine which ports SharePoint Server 2013 and Workflow Manager are using for both HTTP and HTTPS by using IIS Manager as shown in the figure.
Figure: Use IIS Manager to view the ports used by Workflow Manager
Workflow Manager communicates by using TCP/IP or Named Pipes. Make sure that the appropriate communication protocol is enabled on the SQL Server instance that hosts the Workflow Manager databases.
The SQL Browser Service must be running on the SQL Server instance that hosts the Workflow Manager databases.
The System Account cannot be used to develop a workflow.
To troubleshoot Workflow Manager, see Troubleshooting Workflow Management and Execution.
To troubleshoot SharePoint Server 2013, see Troubleshooting SharePoint 2013.
Installation and Deployment for SharePoint 2013 Resource Center
What’s New in SharePoint 2013 Resource Center
-
Installing Workflow Manager certificates in SharePoint Server 2013
Published: September 11, 2012
Summary: Learn how to configure SSL certificates for encrypted communication between Workflow Manager and SharePoint Server 2013.
Applies to: SharePoint Server 2013
Secure Socket Layer (SSL) is an encrypted communication protocol which uses encryption certificates. Workflow Manager and SharePoint Server 2013 can communicate in a secure manor using SSL. This article describes the steps required to setup and configure SSL certificates.
-
Configuration steps
The following sections provide instructions for configuring SSL communication with Workflow Manager and SharePoint Server 2013.
Enable Secure Sockets Layer (SSL) in IIS Manager. For guidance on completing the configuration, see the following:
Under some circumstances, you must obtain and install Workflow Manager “issuer” certificates on SharePoint Server 2013. Here are the circumstances where you must install Workflow Manager certificates:
- If SSL is enabled either on SharePoint Server 2013 (which is not the default) or on Workflow Manager (which is the default), AND
- If SharePoint Server 2013 and Workflow Manager do not share a Certificate Authority, AND
- If Workflow Manager is configured to generate self-signed certificates (which is the default).
Note:
Product trial, workflow development, and troubleshooting are easier if SSL is not enabled. However, communication between SharePoint Server 2013 and Workflow Manager is not encrypted if SSL is not enabled. For this reason, SSL should be enabled for production configurations.
To obtain and export certificates from the Workflow Manager server
- On a computer that has Workflow Manager installed, choose IIS Manager, Sites. Right-click Workflow Management Site, and then choose Edit Bindings.
- Choose the https port, and then choose Edit. Choose the View button in the SSL Certificate section.
-
To export the issuer certificate, do the following:
- In the Certificate window, choose the Certification path tab.
- Select root certification path and choose View.
- On the Details tab, choose Export Certificate, and take the default options in the export wizard.
- Give the exported certificate file a friendly name.
- In the Certificate window, choose the Certification path tab.
To install certificates on SharePoint Server 2013
- Copy the issuer certificate to your SharePoint Server 2013 computer.
- Add the certificates to the Windows Certificate store.
-
For each certificate, do the following:
- Double-click the file to open and view the certificate.
- On the certificate, choose the Install Certificate button to start the installation wizard.
- In the wizard, choose Place all certificates in the following store, and then choose Trusted Root Certification Authorities.
- Double-click the file to open and view the certificate.
- Add the certificates to SharePoint Server by going to the SharePoint Management shell and running the New-SPTrustedRootAuthority cmdlet. Do this for each certificate file.
-
Create a web application in SharePoint 2013
Published: July 16, 2012
Summary: SharePoint 2013 web applications isolate content for specific types of users within your site collections.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
The following downloadable resources, articles on TechNet, and related resources provide information about how to create web applications.
Introduction
A SharePoint 2013 web application is composed of an Internet Information Services (IIS) web site that acts as a logical unit for the site collections that you create. Before you can create a site collection, you must first create a Web application. Each web application is represented by a different IIS web site with a unique or shared application pool. You can assign each web application a unique domain name. This helps prevent cross-site scripting attacks. When you create a new web application, you also create a new content database and define the authentication method used to connect to the database. In addition, you define an authentication method to be used by the IIS Web site in SharePoint 2013.
-
TechNet articles about how to create web applications
The following articles about how to create web applications are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Create web applications that use classic mode authentication in SharePoint 2013
Explains how to create a web application that uses classic mode (Windows-classic) authentication in SharePoint 2013.
Illustrates how to create SharePoint 2013 web applications that use claims-based authentication or classic-mode authentication.
Configure basic authentication for a claims-based web application in SharePoint 2013
Explains how to configure basic authentication for a web application that uses claims-based authentication in SharePoint 2013.
Configure digest authentication for a claims-based web application in SharePoint 2013
Explains how to configure digest authentication for a web application that uses claims-based authentication in SharePoint 2013.
-
Create web applications that use classic mode authentication in SharePoint 2013
Updated: October 2, 2012
Summary: Learn how to create a web application that uses classic mode (Windows-classic) authentication in SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
In SharePoint 2013, claims-based authentication is the default and preferred method of user authentication and is required to take advantage of server-to-server authentication and app authentication. In Central Administration, you can only configure claims-based authentication when you manage web applications. You can also use Windows PowerShell cmdlets. The use of classic mode authentication, also known as Windows classic authentication, is discouraged in SharePoint 2013 and you can only create or configure web applications for classic mode authentication with Windows PowerShell cmdlets.
Important:
Office Web Apps can be used only by SharePoint 2013 web applications that use claims-based authentication. Office Web Apps rendering and editing will not work on SharePoint 2013 web applications that use classic mode authentication. If you migrate SharePoint 2010 web applications that use classic mode authentication to SharePoint 2013, you must migrate them to claims-based authentication to allow them to work with Office Web Apps. For more information, see Use Office Web Apps with SharePoint 2013.
To use Windows claims-based authentication instead (recommended), see Create claims-based web applications in SharePoint 2013. To convert a web application that uses classic mode to use claims-based authentication, see Migrate from classic-mode to claims-based authentication in SharePoint 2013.
Important:
The steps in this article apply to both SharePoint Foundation 2013 and SharePoint Server 2013.
-
Before you begin
Before you perform this procedure, confirm the following:
- Your system is running SharePoint 2013.
-
You have determined the design of your logical architecture.
For more information, see Logical architecture components.
-
You have planned authentication for your web application.
For more information, see Plan for user authentication methods.
- If you use Secure Sockets Layer (SSL), you must associate the SSL certificate with the web application’s IIS website after the IIS website is created. For more information about how to set up SSL, see How to Setup SSL on IIS 7.0 (http://go.microsoft.com/fwlink/p/?LinkId=187887). SSL is required by default for web applications that are used in server-to-server authentication and app authentication scenarios.
-
You understand host-named site collections.
For more information, see Host-named site collections planning.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Your system is running SharePoint 2013.
-
Create a web application that uses classic mode authentication with Windows PowerShell
Perform the following procedure to use Windows PowerShell to create a web application that uses classic mode authentication.
To create a web application that uses classic mode authentication with Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
New-SPWebApplication -Name <Name> -ApplicationPool <ApplicationPool> -AuthenticationMethod <WindowsAuthType> -ApplicationPoolAccount <ApplicationPoolAccount> -Port <Port> -URL <URL>
Where:
- <Name> is the name of the new web application.
- <ApplicationPool> is the name of the application pool.
- < WindowsAuthType > is either “NTLM“ or “Kerberos“. Kerberos is recommended.
- <ApplicationPoolAccount> is the user account that this application pool will run as.
- <Port> is the port on which the web application will be created in IIS.
- <URL> is the public URL for the web application.
-
Example
New-SPWebApplication -Name “Contoso Internet Site” -ApplicationPool “ContosoAppPool” -AuthenticationMethod “Kerberos” -ApplicationPoolAccount (Get-SPManagedAccount “CONTOSO\jdoe”) -Port 80 -URL “https://www.contoso.com”
For more information, see New-SPWebApplication.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
After this procedure is complete, you can create one or more site collections for this web application. For more information, see Create a site collection.
After you successfully create the web application, when you open the Central Administration page, you see a health rule warning that indicates that one or more web applications is enabled with classic authentication mode. This is a reflection of our recommendation to use claims-based authentication instead of classic mode authentication.
-
Create claims-based web applications in SharePoint 2013
Published: July 16, 2012
Summary: Illustrates how to create SharePoint 2013 web applications that use claims-based authentication or classic-mode authentication.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Claims-based authentication is a requirement to enable the advanced functionality of SharePoint 2013. This article explains how to use either Central Administration or Windows PowerShell to create a SharePoint 2013 web application that uses claims-based authentication. Claims-based authentication is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication. However, this article also provides guidance for using Windows PowerShell to create classic-mode web applications if you have a specific scenario that cannot support claims-based authentication. Be aware that classic-mode authentication is deprecated in this release, and it will not be available in the next version. For more information, see Plan for server-to-server authentication in SharePoint 2013
Important:
Secure Sockets Layer (SSL) is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication.
You can create a web application by using the SharePoint Central Administration website or Windows PowerShell. You typically use Windows PowerShell to create a web application. If you want to automate the task of creating a web application, which is common in enterprises, use Windows PowerShell. After you complete the procedure, you can create one or several site collections.
In this article:
- Create a claims-based web application by using Central Administration
- Create a claims-based web application by using Windows PowerShell
- Create a classic-mode web application by using Windows PowerShell
Note:
The steps in this article apply to both SharePoint Foundation 2013 and SharePoint Server 2013.
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Create a claims-based web application by using Central Administration
-
Create a claims-based web application by using Central Administration
Use the procedure described in this section to create a new claims-based SharePoint 2013 web application using the Central Administration.
To create a claims-based web application by using Central Administration
- Verify that you have the following administrative credentials:
- To create a web application, you must be a member of the Farm Administrators SharePoint group.
- Start SharePoint 2013 Central Administration.
-
For Windows Server 2008 R2:
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Central Administration.
If SharePoint 2013 Central Administration is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Central Administration.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- On the Central Administration Home page, click Application Management.
- On the Application Management page, in the Web Applications section, click Manage web applications.
- In the Contribute group of the ribbon, click New.
- On the Create New Web Application page, in the IIS Web Site section, you can configure the settings for your new web application by selecting one of the following two options:
- Click Use an existing IIS web site, and then select the web site on which to install your new web application.
- Click Create a new IIS web site, and then type the name of the web site in the Name box.
-
In the Port box, type the port number you want to use to access the web application. If you are using an existing web site, this field contains the current port number.
Note:
The default port number for HTTP access is 80, and the default port number for HTTPS access is 443.
-
Optional: In the IIS Web Site section, in the Host Header box, type the host name (for example, http://www.contoso.com) that you want to use to access the web application.
Note:
You do not need to populate this field unless you want to configure two or more IIS web sites that share the same port number on the same server, and DNS has been configured to route requests to the same server.
- In the Path box, type the path to the IIS web site home directory on the server. If you are creating a new web site, this field contains a suggested path. If you are using an existing web site, this field contains the current path of that web site.
- In the Security Configuration section, choose whether or not to Allow Anonymous access and whether or not to Use Secure Sockets Layer (SSL).
Important:
Secure Sockets Layer (SSL) is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication. In general, we strongly recommend using SSL for web applications.
-
In the Security Configuration section, click Yes or No for the Allow Anonymous options. If you choose to Yes, visitors can use the computer-specific anonymous access account (that is, IIS_IUSRS) to access the web site.
Note:
If you want users to be able to access any site content anonymously, you must enable anonymous access for the entire web application zone before you enable anonymous access at the SharePoint site level. Later, site owners can configure anonymous access for their sites. If you do not enable anonymous access at the web application level, site owners cannot enable anonymous access at the site level.
- In the Security Configuration section, click Yes or No for the Use Secure Sockets Layer (SSL) options. If you choose Yes, you must request and install an SSL certificate to configure SSL. For more information about how to set up SSL, see How to Setup SSL on IIS 7.0.
- In the Claims Authentication Types section, select the authentication method that you want to use for the web application.
-
To enable Windows authentication, select Enable Windows Authentication and, in the drop-down menu, select NTLM or Negotiate (Kerberos). We recommend using Negotiate (Kerberos).
If you do not want to use Integrated Windows authentication, clear Integrated Windows authentication.
Note:
If you do not select Windows Authentication for at least one zone of this web application, crawling for this web application will be disabled.
-
If you want users’ credentials to be sent over a network in a nonencrypted form, select Basic authentication (credentials are sent in clear text).
Note:
You can select basic authentication or integrated Windows authentication, or both. If you select both, SharePoint 2013 offers both authentication types to the client web browser. The client web browser then determines which type of authentication to use. If you only select Basic authentication, ensure that SSL is enabled. Otherwise, a malicious user can intercept credentials.
-
To enable forms-based authentication, select Enable Forms Based Authentication (FBA), and then enter the ASP.NET Membership provider name and the ASP.NET Role manager name.
Note:
If you select this option, ensure that SSL is enabled. Otherwise, a malicious user can intercept credentials.
- If you have set up Trusted Identity Provider authentication by using Windows PowerShell, the Trusted Identity provider check box is selected.
- In the Sign In Page URL section, choose one of the following options to sign into SharePoint 2013:
- Select Default Sign In Page URL to redirect users to a default sign-in web site for claims-based authentication.
- Select Custom Sign In page URL and then type the sign-in URL to redirect users to a customized sign-in web site for claims-based authentication.
-
In the Public URL section, type the URL for the domain name for all sites that users will access in this web application. This URL will be used as the base URL in links that are shown on pages within the web application. The default URL is the current server name and port, and it is automatically updated to reflect the current SSL, host header, and port number settings on the page. If you deploy SharePoint 2013 behind a load balancer or proxy server, then this URL may need to be different than the SSL, host header, and port settings on this page.
The Zone value is automatically set to Default for a new web application. You can change the zone when you extend a web application.
- In the Application Pool section, do one of the following:
- Click Use existing application pool, and then select the application pool that you want to use from the drop-down menu.
- Click Create a new application pool, and then type the name of the new application pool, or keep the default name.
- Click Predefined to use a predefined security account for this application pool, and then select the security account from the drop-down menu.
-
Click Configurable to specify a new security account to be used for an existing application pool.
Note:
To create a new account, click the Register new managed account link.
- In the Database Name and Authentication section, choose the database server, database name, and authentication method for your new web application, as described in the following table.
|
Action |
|
Database Server |
Type the name of the database server and SQL Server instance you want to use in the format <SERVERNAME\instance>. You can also use the default entry. |
|
Database Name |
Type the name of the database, or use the default entry. |
|
Database Authentication |
Select the database authentication to use by doing one of the following:
SQL authentication sends the SQL authentication password to SQL Server in an unencrypted format. We recommend that you only use SQL authentication if you force protocol encryption to SQL Server to encrypt your network traffic by using IPsec. |
- If you use database mirroring, in the Failover Server section, in the Failover Database Server box, type the name of a specific failover database server that you want to associate with a content database
- In the Service Application Connections section, select the service application connections that will be available to the web application. In the drop-down menu, click default or [custom]. You use the [custom] option to choose the service application connections that you want to use for the web application.
- In the Customer Experience Improvement Program section, click Yes or No.
- Click OK to create the new web application.
-
Create a claims-based web application by using Windows PowerShell
Use the procedure in this section to create a new claims-based SharePoint 2013 web application using Windows PowerShell.
To create a claims-based web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
-
You must read about_Execution_Policies.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Permissions and Add-SPShellAdmin.
-
To create a claims-based authentication provider, from the Windows PowerShell command prompt, type the following:
$ap = New-SPAuthenticationProvider
-
To create a claims-based web application, from the Windows PowerShell command prompt, type the following:
New-SPWebApplication -Name <Name>
-ApplicationPool <ApplicationPool>
-ApplicationPoolAccount <ApplicationPoolAccount>
-URL <URL> -Port <Port> -AuthenticationProvider $ap
Where:
- <Name> is the name of the new web application that uses claims-based authentication.
- <ApplicationPool> is the name of the application pool.
- <ApplicationPoolAccount> is the user account that this application pool will run as.
- <URL> is the public URL for this web application.
-
<Port> is the port on which the web application will be created in IIS.
Note:
For more information, see New-SPWebApplication.
The following example creates an https claims-based web application, using the current user credentials and the current machine name:
$waUrl = “https://” + $env:ComputerName
$siteAdmin = $env:userdomain + “\” + $env:username;
CreateWindowsWebApp -url $waUrl -title “WinClaimsInbound” -site_admin $siteAdmin -app_pool_name “WebAppPool1” -app_pool_account $siteAdmin -use_claims –use_ssl;
Note:
After you have created the web site, you must configure SSL in IIS for this newly created web site. For more information about how to set up SSL, see How to Setup SSL on IIS 7.0.
If you want your web application to use HTTP, do not use the –use_ssl parameter, and use the http scheme for the –url parameter.
-
Create a classic-mode web application by using Windows PowerShell
Use the procedure in this section to create a new classic-mode SharePoint 2013 web application using Windows PowerShell.
To create a classic-mode web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
- You must read about_Execution_Policies.
-
From the Windows PowerShell command prompt, type the following:
New-SPWebApplication –Name <Name>
–ApplicationPool <ApplicationPool>
-AuthenticationMethod <WindowsAuthType>
–ApplicationPoolAccount <ApplicationPoolAccount>
-Port <Port> -URL <URL>
Where:
- <Name> is the name of the new web application that uses classic-mode authentication.
- <ApplicationPool> is the name of the application pool.
- <WindowsAuthType> is either “NTLM“ or “Kerberos“. Kerberos is recommended.
- <ApplicationPoolAccount> is the user account that this application pool will run as.
- <Port> is the port on which the web application will be created in IIS.
-
<URL> is the public URL for the web application.
Note:
For more information, see New-SPWebApplication.
Note:
After you successfully create the web application, when you open the Central Administration page, you see a health rule warning that indicates that one or more web applications is enabled with classic authentication mode. This is a reflection of our recommendation to use claims-based authentication instead of classic mode authentication.
-
Configure basic authentication for a claims-based web application in SharePoint 2013
Published: September 25, 2012
Summary: Learn how to configure basic authentication for a web application that uses claims-based authentication in SharePoint 2013.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Foundation 2013 | SharePoint Server 2013 Standard
You can configure basic authentication for one or more zones in a SharePoint 2013 claims-based web application. A web application is an Internet Information Services (IIS) web site that SharePoint 2013 creates and uses. Zones represent different logical paths for gaining access to the network services that are available within the same web application. Within each web application, you can create up to five zones. A different web site in IIS represents each zone. Use zones to enforce different access and policy conditions for large groups of users. To configure basic authentication for one or more zones in a SharePoint 2013 web application, use IIS Manager console, instead of SharePoint Central Administration.
-
Before you begin
Before you perform this procedure, confirm the following:
- Your system is running SharePoint 2013.
- Basic authentication requires previously assigned Windows account credentials for user access.
-
You understand basic authentication for web traffic.
Basic authentication enables a web browser to provide credentials when the browser makes a request during an HTTP transaction. Because user credentials are not encrypted for network transmission but are sent over the network in plaintext, we do not recommend that you use basic authentication over an unsecured HTTP connection. To use basic authentication, you should enable Secure Sockets Layer (SSL) encryption for the web site; otherwise, the credentials can be intercepted by a malicious user.
Note:
Because SharePoint 2013 runs as websites in IIS, administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support (http://go.microsoft.com/fwlink/p/?LinkId=246502)
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products (http://go.microsoft.com/fwlink/p/?LinkId=246501)
- Keyboard shortcuts (http://go.microsoft.com/fwlink/p/?LinkID=246504)
- Touch (http://go.microsoft.com/fwlink/p/?LinkId=246506)
- Your system is running SharePoint 2013.
-
Configure IIS to enable basic authentication
Use the IIS Manager console to configure IIS to enable basic authentication for one or more of the following zones for a claims-based web application:
- Default
- Intranet
- Extranet
The Default zone is the zone that is first created when a web application is created. The other zones are created by extending a web application. For more information, see Extend a claims-based web application.
To configure IIS to enable basic authentication
- Default
- Verify that you a member of the Administrators group on the server on which you are configuring IIS.
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) Manager to start IIS Manager console.
- Expand Sites in the console tree, and then click the IIS web site that corresponds to the web application zone on which you want to configure basic authentication.
- In Features View, in IIS, double-click Authentication.
- In Features View, in Authentication, right-click Basic Authentication, and then click Enable.
- Right-click Basic Authentication, and then click Edit.
-
In the Edit Basic Authentication Settings dialog box, in the Default domain text box, type the appropriate default domain.
The default domain is the name of a domain against which you want users to be authenticated when they do not provide a domain name.
-
In the Realm text box, type the appropriate realm, and then click OK.
The realm is a DNS domain name or an IP address that will use the credentials that are authenticated against your internal Windows domain. Configuring a realm name for basic authentication is optional.
The web site is now configured to use basic authentication.
You can also configure basic authentication when you create a web application in SharePoint Central Administration by selecting Basic authentication (password is sent in clear text) in the Claims Authentication Types section of the Create New Web Application dialog box. For more information, see Create claims-based web applications in SharePoint 2013.
Security
In the Claims Authentication Types section of the Create New Web Application dialog box, you can select Integrated Windows authentication, Basic authentication (password is sent in clear text), or both. If you select both, SharePoint 2013 will offer both authentication types to the client web browser. The client web browser then determines the type of authentication to use. If you only select Basic authentication (password is sent in clear text), make sure that you enable SSL for this web application.
-
Configure digest authentication for a claims-based web application in SharePoint 2013
Published: September 25, 2012
Summary: Learn how to configure digest authentication for a web application that uses claims-based authentication in SharePoint 2013.
Applies to: SharePoint Server 2013 Enterprise | SharePoint Server 2013 Standard | SharePoint Foundation 2013
You can configure digest authentication for one or more zones in a SharePoint 2013 claims-based web application. A web application is an Internet Information Services (IIS) web site that SharePoint 2013 creates and uses. Zones represent different logical paths for gaining access to the same web application. Within each web application, you can create up to five zones. A different web site in IIS represents each zone. Use zones to enforce different access and policy conditions for large groups of users. To configure digest authentication for one or more zones in a SharePoint 2013 web application, use IIS Manager console, instead of SharePoint Central Administration.
Unlike basic authentication, digest authentication encrypts user credentials to increase security. User credentials are sent as an MD5 message digest in which the original user name and password cannot be determined. Digest authentication uses a challenge/response protocol that requires the authentication requestor to present valid credentials in response to a challenge from the server. To authenticate against the server, the client has to supply an MD5 message digest in a response that contains a shared secret password string. The MD5 Message-Digest Algorithm is described in RFC 1321. For access to RFC 1321, see The Internet Engineering Task Force (http://go.microsoft.com/fwlink/p/?LinkId=159913).
-
Before you begin
Before you perform this procedure, confirm the following:
- Your system is running SharePoint 2013.
- The user and IIS server must be members of, or trusted by, the same domain.
- Users must have a valid Windows user account stored in Active Directory Domain Services (AD DS) on the domain controller.
- The domain must use a Windows Server 2008 or Windows Server 2008 R2 domain controller.
-
You understand digest authentication for web traffic.
For more information, see What is Digest Authentication? (http://go.microsoft.com/fwlink/p/?LinkId=209085).
Note:
Because SharePoint 2013 runs as websites in IIS, administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support (http://go.microsoft.com/fwlink/p/?LinkId=246502)
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products (http://go.microsoft.com/fwlink/p/?LinkId=246501)
- Keyboard shortcuts (http://go.microsoft.com/fwlink/p/?LinkID=246504)
- Touch (http://go.microsoft.com/fwlink/p/?LinkId=246506)
- Your system is running SharePoint 2013.
-
Configure IIS to enable digest authentication
Use IIS Manager console to configure IIS to enable digest authentication for one or more of the following zones for a claims-based web application:
- Default
- Intranet
- Extranet
The Default zone is the zone that is first created when a web application is created. The other zones are created by extending a web application. For more information, see Extend a claims-based web application.
To configure IIS to enable digest authentication
- Default
- Verify that you are a member of the Administrators group on the server on which you are configuring IIS.
- Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) Manager to start IIS Manager console.
- Expand Sites in the console tree, and then click the IIS web site that corresponds to the web application zone on which you want to configure digest authentication.
- In Features View, in IIS, double-click Authentication.
- In Features View, in Authentication, right-click Digest Authentication, and then click Enable.
- Right-click Digest Authentication, and then click Edit.
-
In the Edit Digest Authentication Settings dialog box, in the Realm text box, type the appropriate realm, and then click OK.
The realm is a DNS domain name or an IP address that will use the credentials that have been authenticated against your internal Windows domain. You must configure a realm name for digest authentication.
The web site is now configured to use digest authentication.
-
Install and manage solutions for SharePoint 2013
Published: October 16, 2012
Summary: Learn how to install solutions or components that were customized by developer or web designers to a SharePoint 2013 environment.
Applies to: SharePoint Server 2013 | SharePoint Foundation 2013
The process to install custom site elements and solution packages in SharePoint 2010 Products and SharePoint 2013 has not changed significantly. Detailed content about how to install custom elements is available in the SharePoint Server 2010 technical library in the following section Deploy customizations – overview (SharePoint Server 2010). To install solution packages, you can use a new parameter, CompatibilityLevel, to install the solution to the latest version directories, or to use only the current version tracked in the cab file. For more information, see Install-SPSolution.
To add functionality to SharePoint sites in SharePoint 2013, you can use apps for SharePoint. For more information about apps for SharePoint, see Install and manage apps for SharePoint 2013.
-
TechNet articles about how to install and manage solutions
The following articles about how to install and manage solutions are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Overview article and detailed content about how to install custom elements for SharePoint Server 2010. Applies to both SharePoint 2010 Products and SharePoint 2013.
Content about how to manage sandboxed solutions for SharePoint Server 2010. Applies to both SharePoint 2010 Products and SharePoint 2013.
List Windows PowerShell cmdlets that help you manage features and solutions in a SharePoint 2013 farm.
-
Additional resources about how to install and manage solutions
The following resources about how to install and manage solutions are available from other subject matter experts.
Content
Description
Visit the TechCenter to access videos, community sites, documentation, and more.
Find resources for building solutions on SharePoint 2013. This includes how-to articles, code samples, and the SharePoint SDK.
Read draft content, learn about new content, and join the conversation at this IT pro content team blog.
Learn how to get the most out of SharePoint sites by reading this blog.
-
Install and manage apps for SharePoint 2013
Published: July 16, 2012
Summary: The following resources provide information about apps for SharePoint, the SharePoint Store, and the App Catalog and how to install, manage, and monitor apps.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
-
Downloadable resources about apps for SharePoint
Download the following content for information about apps for SharePoint.
Content
Description
Model poster describing apps from an IT Pro perspective.
-
TechNet articles about apps for SharePoint
The following articles about apps are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Read an overview of apps for SharePoint from an IT Pro perspective.
Understand what you need to plan for before you support apps for SharePoint.
Learn how to plan for app authentication in SharePoint 2013.
App permissions management enforces security and provides the additional functionality that apps enable on SharePoint 2013 sites.
Follow these steps to configure your environment to support apps for SharePoint.
Learn how to configure app authentication in SharePoint 2013.
Follow these steps to configure and manage the App Catalog and configure SharePoint Store rules.
Learn how to install an app for SharePoint from the App Catalog or SharePoint Store.
Learn how to remove an app for SharePoint from a site collection.
Follow these steps to specify which apps for SharePoint to monitor, and then monitor the apps.
Learn how SharePoint farm administrators assign, monitor, and manage the app for SharePoint licenses in SharePoint Server 2013.
-
Additional resources about apps for SharePoint
The following resources about apps for SharePointare available from other subject matter experts.
Content
Description
Installation and Deployment for SharePoint 2013 Resource Center
Visit the Resource Center to access videos, community sites, documentation, and more.
Content on MSDN
Articles about apps for SharePoint in the SharePoint Server 2013 Software Development Kit
-
Overview of apps for SharePoint 2013
Published: July 16, 2012
Summary: The apps for SharePoint are a powerful, easy way to add functionality to a SharePoint site. Understand how they work, how they are integrated with SharePoint sites, and how they are isolated from your site content.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
The apps for SharePoint provide a new method to deliver specific information or functionality to a SharePoint site. An app for SharePoint is a small, easy-to-use, stand-alone app that solves a specific end-user or business need. Site owners can discover and download apps for SharePoint from a public SharePoint Store or from their organization’s internal App Catalog and install them on their SharePoint sites. These apps for SharePoint integrate the best of the web with SharePoint 2013. They do not replace SharePoint features and solution packages, which customize or enhance SharePoint sites. Unlike features and solutions, which farm or site collection administrators have to install, apps for SharePoint are stand-alone applications that owners of sites can add to their SharePoint sites. The apps for SharePoint have a simple lifecycle – they can be installed, upgraded, and uninstalled by site owners.
The following are examples of apps for SharePoint that site owners could add to their sites:
- An app that provides event planning tools.
- An app that provides a shopping cart experience for a site.
- An app that sends a note of recognition for good work (kudos) to someone in the organization.
Microsoft will host and control a public SharePoint Store, where developers will be able to publish and sell their custom apps for SharePoint. End users and IT professionals will be able to obtain these custom apps for personal or corporate use. This SharePoint Store will handle the end-to-end acquisition experience from discovery to purchase, upgrades, and updates.
Company-developed and approved apps can also be deployed to an organization’s internal App Catalog hosted on SharePoint 2013 or SharePoint Online. This controls the visibility of apps within organizations.
Important:
This article applies to both SharePoint Foundation 2013 and SharePoint Server 2013.
- An app that provides event planning tools.
-
Where are apps for SharePoint hosted?
There are several options for hosting apps for SharePoint.
- Provider-hosted
- Hosted in the cloud (Windows Azure autohosted)
- Hosted in a SharePoint environment
- Several combinations of these options.
Depending on the hosting option, the app can contain different elements and take advantage of different components.
Illustration of hosting options for apps for SharePoint
No matter the hosting option for the app, if you want users to be able to install and use apps for SharePoint in your environment, you will have to configure your environment to support them.
For more information about hosting options, see the SharePoint 2013 developer documentation.
- Provider-hosted
-
How are apps for SharePoint and SharePoint sites related?
Site owners can add apps for SharePoint to their sites. If an app contains SharePoint components, those components are stored in a subweb of the site that is automatically created when you install the app. Apps have their own, isolated URLs, which are separate from the URL for the site that contains the app. If the app is a Provider-hosted or Windows Azure autohosted app, the app components are stored in those locations. For example, in the following diagram, App1 contains custom business logic and is stored on an external server – it is an Windows Azure autohosted app and does not store content in a subweb of the site. App2 is a SharePoint hosted app with only SharePoint components. App2’s content is stored in a subweb of the site on which it is installed.
Illustration of relationship between apps for SharePoint and SharePoint sites
-
What is the URL for an app for SharePoint?
By default, apps are deployed to their own web site in a special, isolated domain name, instead of in the same domain name as your farm. Processes run under that domain name and do not affect the SharePoint sites. This difference in domain names provides a layer of isolation for the apps. The use of a different domain name from the SharePoint sites prevents cross-site scripting between the apps and sites and unauthorized access to users’ data.
Each installation of an app has a unique URL in your environment. You determine the template for that URL (by specifying a domain name and an app prefix), and then app URLs are automatically generated based on that template. Paths for the apps are based on the URL for the site where they are installed. When you install an app to a site, a subweb of that site is created to host the app content. The subweb for the app is hierarchically below the site collection, but has an isolated unique host header instead of being under the site’s URL. The following diagram shows the relationship between the site’s URL and the app’s URL:
Illustration of URL for an app for SharePoint
In this diagram, the Main SharePoint Site is the site on which the user installed the app. The App1 SharePoint Site is a subweb of the Main site that contains the app and its components. The URL for the App1 SharePoint site is based on that of the Main SharePoint site. However, it is in a different domain, has a prefix-apphash at the beginning, and has an app name at the end for the subweb name. The prefix-apphash part of the URL is designed to support multi-tenant environments. In a multi-tenant environment, each tenant has its name that is combined with the apphash to provide a unique domain name for the app. If you are not in a multi-tenant environment, you can use the same app prefix for all URLs.
-
Use and benefits of apps for SharePoint
The apps for SharePoint allow users to add quick functionality to their site without your intervention. Unlike templates, features, and solutions, which an IT administrator must deploy, site owners can add apps for SharePoint to their sites or remove them. Because apps for SharePoint are limited in scope to a subweb and have an isolated URL, they do not interact with the rest of your farm or open your environment to cross-site scripting risks.
Your organization can develop its own apps for SharePoint. You can make apps for SharePoint available from the SharePoint Store, and you can make these apps available in the App Catalog so that users know which apps for SharePoint are approved for use in your environment. Users can easily update apps for SharePoint with new versions when they become available.
-
Impacts of apps for SharePoint
Supporting apps for SharePoint in your environment does require a configuration change to your environment. There are two main considerations:
- Requirements for supporting apps for SharePoint You must be running the Subscription Settings and App Management service applications to use apps for SharePoint. You must create the DNS domain to contain the URLs for apps for SharePoint in your environment.
- Plan for capacity Each app for SharePoint that is installed creates a subweb under the site on which it is installed with its own URL. This means that environments that contain many apps for SharePoint will have many additional subwebs. Be sure to consider this when planning for capacity for your farm.
For more information about these and other considerations, see Plan for apps for SharePoint 2013 and Configure an environment for apps for SharePoint 2013.
- Requirements for supporting apps for SharePoint You must be running the Subscription Settings and App Management service applications to use apps for SharePoint. You must create the DNS domain to contain the URLs for apps for SharePoint in your environment.
-
Plan for apps for SharePoint 2013
Published: July 16, 2012
Summary: Determine your environment’s app for SharePoint policy, whether to use the App Catalog, the URLs to use for apps for SharePoint, and the settings to configure for apps for SharePoint in your environment.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The apps for SharePoint provide a new method to deliver specific information or functionality to a SharePoint site. An app for SharePoint is a small, easy-to-use, stand-alone productivity app that solves a specific end-user need. Before you allow users to install apps in a SharePoint environment, you must plan how you want to support them. You have to determine your organization’s policy around apps for SharePoint, plan your configuration settings, and determine how to manage and monitor the apps. This article explains key decisions and helps you understand the choices to make as you plan to support apps for SharePoint.
In this article:
- Governance: determine the app for SharePoint policy for your organization
- Plan app configuration settings
- Plan App Catalog
- Plan to monitor apps
- Plan for app licenses
For more conceptual information about apps for SharePoint, see Overview of apps for SharePoint 2013.
- Governance: determine the app for SharePoint policy for your organization
-
Governance: determine the app for SharePoint policy for your organization
The first decision about apps for SharePoint is the extent to which you want to use them in your organization and the policy for using them. The following questions can help you frame your discussion about your policy:
-
Do you want users to be able to install and use apps for SharePoint?
- If so, keep reading and shaping your policy.
- If not, then you can use settings in the SharePoint Store to control whether users can get apps from the SharePoint Store. You cannot block users from viewing the SharePoint Store. However, you can prevent them from purchasing or downloading apps for SharePoint. If you choose to prevent users from getting apps for SharePoint from the SharePoint Store, you should create a policy statement and notify users to advise users that you have chosen not to support apps for SharePoint. Otherwise, the only notification to users will be an error when they try to install an app for SharePoint.
- If so, keep reading and shaping your policy.
-
Do you want to restrict or control the apps for SharePoint that users can install and use?
If so, you can do one or both of the following:
- Set up an App Catalog to provide a set of apps for SharePoint that users can install and use.
- Use the App Request feature to control the purchasing and licensing of apps for SharePoint.
- Set up an App Catalog to provide a set of apps for SharePoint that users can install and use.
-
In which environments do you want apps for SharePoint to be available? For example, intranet, extranet, or Internet environments?
You should decide which environments are appropriate for using apps for SharePoint and configure the settings for apps for SharePoint only in those farms or for those web applications.
-
Do you want to control who can purchase apps for SharePoint?
No explicit setting prevents a user from purchasing an app for SharePoint. However, you can create a request process that requires users to submit a request that your organization reviews to ensure that appropriate persons make purchases.
-
Do you want to control who can install apps for SharePoint?
A user must have the Manage Web site and Create Subsites permissions to install an app for SharePoint. By default, these permissions are available only to users who have the Full Control permission level or who are in the Site Owners group.
-
Do you want to control which hosting options can be used for apps for SharePoint in your environment?
You cannot explicitly specify the hosting options (Provider-hosted, Windows Azure autohosted, or SharePoint hosted) that are available. The creator of an app determines how it is hosted based on what the app contains or does. However, you can choose a specific hosting option for apps for SharePoint that you provide for your own organization in the App Catalog.
-
How many and which apps for SharePoint do you want to monitor?
You must explicitly select the apps for SharePoint to monitor for a farm.
Note:
The ability to monitor apps for SharePoint is not available for SharePoint Foundation 2013.
-
-
Plan app configuration settings
Before you use apps for SharePoint in your environment, you have to configure your environment to support them. If you don‘t configure your environment, users who try to install and use apps for SharePoint receive error messages. For all apps for SharePoint, you must set up a Domain Name Services (DNS) domain name to provide a host name for the installed apps. By using a separate domain name, apps for SharePoint are separated from SharePoint sites to prevent unauthorized access to user data and to reduce the likelihood of cross-site scripting attacks. Using separate URLs for apps for SharePoint and SharePoint sites is called app isolation. You also need a DNS record so that the domain name can get correctly resolved. You can create one of two of the following types of DNS records for app for SharePoint URLs:
- A wildcard Canonical Name (CNAME) record that points to the host domain assigned to the SharePoint farm.
- An A record to point to the IP address for the SharePoint farm.
Choose the type of record to point from the app domain to the SharePoint farm domain.
Illustration of a type of record that points from the app domain to a SharePoint farm domain
In addition to setting up the DNS domain to support apps in your environment, you also have to configure the following:
-
SSL Certificates (if they are required)
If you are using SSL to help secure traffic, you must create a wildcard certificate to use for all app URLs. Wildcard certificates cost about the same as five individual certificates. So, you can quickly justify the cost of a wildcard if you expect to use more than five instances of apps for SharePoint in your environment.
Important:
Each instance of an app for SharePoint that is installed has its own URL. Therefore, if you only have one app for SharePoint in your environment, but the app is installed on six different sites, then you will have six different app URLs.
-
The Subscription Settings and App Management service applications
These services support apps in your environment by storing the data needed to run apps in the farm. The Subscription Settings service stores the tenant name and the App Management Service stores app licenses, app principals, app users, app registrations, and so on.
-
The app URLs to use
Each app has a unique URL in your environment. You determine the template for that URL, and then app URLs are automatically generated by using the prefix and domain that you specify. For example, prefix–Apphash.ContosoApps.com/sites/web1/appname.
The Configure an environment for apps for SharePoint 2013 article explains how to configure these settings.
-
Determine the domain name to use
Each app for SharePoint has a unique URL, which is made up of the app domain and a prefix for that domain that consists of a tenant name and an Apphash. The format is as follows: prefix–Apphash.domain.com. The Apphash is an arbitrarily-assigned unique identifier for each app for SharePoint.
Note:
The examples in this section use an Internet-style domain name for clarity. This does not imply that you must expose your apps to the Internet or that your URLs must use this format. Note that for actual hosting environments you must still organize a DNS routing strategy within your intranet and optionally configure your firewall.
When you choose the domain name and prefixes to use for your environment, consider the following:
-
Consider using a unique domain name, not a subdomain
To help improve security, the domain name that you choose should not be a subdomain of the root domain name that hosts other applications (other than SharePoint sites). This is because other applications that run under that host name might contain sensitive information stored in cookies that might not be protected. Code can set or read cookies across different domains that are under the same domain. A malicious developer could use code in an app for SharePoint to set or read information in a cookie on the root domain from the app for SharePoint subdomain. If a malicious app accessed that cookie information, then you could have an information leak. SharePoint sites have protections against this issue. However, it is still a good idea to use a domain for apps that is separate from your other domains. For example, if the SharePoint sites are at Contoso.com, do not use Apps.Contoso.com. Instead use a unique name such as Contoso-Apps.com. This is not to say that you should never use a subdomain if you have business reasons to do this. However, consider all potential security risks.
-
For single tenant environments, use the same prefix for all apps
If your environment is only used for your own organization and does not host SharePoint sites for other organizations, you configure a prefix that all app for SharePoint URLs in your environment use. For example, you can set the prefix to a word like default so that each app for SharePoint has a URL such as default-Apphash.Contoso-Apps.com.
-
For multi-tenancy environments, use unique prefixes for each tenant’s apps
If your environment has multiple tenants (in other words, you host SharePoint sites for multiple clients), you must be able to identify the URLs that each tenant or client in your environment uses. So, you set the URL prefix to indicate the client’s name or the client’s site’s name. For example, suppose that A. Datum Corporation hosts SharePoint sites for Fabrikam and Fourth Coffee under the adatum.com hosting domain (for example, Fabrikam.Adatum.com and FourthCoffee.Adatum.com). You should set the prefix for your hosted sites so that they are created as Fabrikam-Apphash.AdatamApps.com and FourthCoffee-Apphash.AdatamApps.com.
Important:
If you are in a multi-tenancy environment, you must use Windows PowerShell to configure the URL and prefix.
-
Keep prefixes short and simple
Prefixes must be less than 48 characters and cannot contain special characters or dashes.
-
All URLs will be in the default zone
You cannot use multiple zones in alternate access mapping for apps for SharePoint. By default, all apps for SharePoint are in the intranet zone.
- A wildcard Canonical Name (CNAME) record that points to the host domain assigned to the SharePoint farm.
-
Plan App Catalog
If you decide to provide trusted apps for SharePoint for users to install, you can configure the App Catalog to contain those apps for SharePoint. The App Catalog is a special site that contains the apps for SharePoint that users can install. You can have multiple App Catalogs in your farm, one for each Web application in your farm. To configure the App Catalog for a Web application, you just have to supply the names of the site collection administrators who you want to use for the App Catalog site. After you create the App Catalog, the site collection administrators can upload apps for SharePoint to it.
To plan App Catalog settings, determine the following:
-
Which web applications will need an App Catalog.
This goes together with your decisions about your policy for supporting apps for SharePoint in the SharePoint environment. If you have different types of sites (intranet, extranet, Internet) on different web applications in your farm, you can determine whether you want an App Catalog for each of those web applications.
You can also share an App Catalog site across web applications in a farm. To do this, you configure the App Catalog for one web application, and then configure the App Catalog settings for another web application to use the same URL.
-
Who to add as the site collection administrators for the App Catalog.
The App Catalog is a site inside a web application that can only be accessed from a link in Central Administration or directly by using the URL.
-
-
Plan to monitor apps
Note:
This section applies only to SharePoint Server 2013.
The Monitor Apps page in SharePoint on-premises only lists apps that the Farm administrator adds. The maximum number of apps that can be monitored is limited to 100. Apps that exceed the threshold require a farm administrator to remove existing apps from the list. The first 50 apps appear in the monitoring apps list so the administrator must filter the apps to monitor the rest of the apps.
The apps for SharePoint monitoring data is stored in the SQL Server Usage database as WSS_Logging.
-
Plan for app licenses
SharePoint 2013 does not enforce app licenses. Developers who build apps must add code that retrieves license information and then addresses users. SharePoint 2013 provides the storage and together with SharePoint Store web services the app license renewal. SharePoint Store handles payments for the licenses, issues the correct licenses, and provides the process to verify license integrity. Note that licensing only works for apps that are distributed through the SharePoint Store. Apps that you purchase from another source and apps that you develop internally must implement their own licensing mechanisms. SharePoint 2013 supports the following app licenses formats:
License Type
Duration
User Limit
Free
Perpetual
Unlimited
Trial
30, 60, 120 Days, or Unlimited
Number per user or Unlimited
Paid per user
Perpetual
Number per user
Paid unlimited users (site license)
Perpetual
Unlimited
-
Plan app permissions management in SharePoint 2013
Published: July 16, 2012
Summary: App permissions management enforces security and enables the additional functionality that apps provide on SharePoint 2013 sites.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
The purpose of app permissions management is to manage the ability of apps to access and use internal SharePoint 2013 resources and perform tasks on behalf of users. For app authentication, SharePoint 2013 relies on a trusted token service named the Windows Azure Access Control Service (ACS) to issue time- and scope-limited access tokens for apps. In SharePoint 2013, the ACS acts as the app identity provider. The app authentication process verifies a claim that an app makes and asserts that the app can act on behalf of an authenticated SharePoint 2013 user. The authorization process verifies that an authenticated app has permission to access a specified resource and perform a defined function. You can configure SharePoint 2013 to process app permissions and enable anyone who enters an anonymous web site, for example, to view and fill out a form that requests additional information about a product or service. You can also allow a SharePoint 2013 site owner, or a user with elevated permissions, to purchase and install an app that a defined set of internal SharePoint 2013 users can access. For example, a site owner can purchase and install an expense report app for a workgroup, and the members of the workgroup can use the app to access data in SharePoint 2013 document libraries and generate expense reports. For more information about apps, see Overview of apps for SharePoint 2013.
In this article:
- App permission request scopes
- App permission requests
- App authorization policies
To plan the management of SharePoint 2013 app permissions, you have to determine the specific SharePoint 2013 resources that the app will need to access, and where those resources reside. You also have to determine the minimum permission level that will be required to enable the app to function correctly. In addition, you have to determine the appropriate app authorization policy to ensure that the app functions correctly and complies with specified authorization requirements.
- App permission request scopes
-
Introduction
SharePoint 2013 apps provide a wide variety of powerful tools and enhanced functionality that can increase the usefulness of your SharePoint 2013 deployment. When you decide to support the implementation of SharePoint 2013 apps within your deployment, you need to determine who will be installing the apps and who will be using them. You also need to determine the appropriate scope and permission levels for each app, based on how the app is intended to be used.
This article explains how to decide which scope, permission level, and authorization policy to use for the various types of SharePoint 2013 apps you plan to deploy, depending on how the app is going to be used and who is going to use the app. This article does not explain how to create or configure SharePoint 2013 apps.
-
App permission request scopes
SharePoint 2013 apps use app permission request scopes and permission requests to specify the level at which the app is intended to run, and the permission level that is assigned to the app. The app permission request scope indicates the location within the SharePoint 2013 hierarchy where a permission request will apply. SharePoint 2013 supports the following permission request scopes:
- SPSite Defines the app permission request scope as a SharePoint 2013 site collection.
- SPWeb Defines the app permission request scope as a SharePoint 2013 web site.
- SPList Defines the app permission request scope as a SharePoint 2013 list.
- Tenancy Defines the app permission request scope as a SharePoint 2013 tenancy.
If an app is granted permission to one scope, the permission also applies to the children of that scope. For example, if an app is granted permission to a web site by using the SPWeb scope, the app is also granted permission to each list (SPList scope) that is contained within the SPWeb scope and all list items within each list. Because permission requests are made without information about the topology of the site collection where the app is installed, the scope is expressed as a type rather than as the URL of a specific instance. These scope types are expressed as URIs. Content database related permissions are organized under this URI: http://sharepoint/content. The following table provides an URI example for each app permission request scope.
Scope
URI
SPSite
SPWeb
SPList
Tenancy
- SPSite Defines the app permission request scope as a SharePoint 2013 site collection.
-
App permission requests
App permission requests are collections of permissions that enable apps to perform specific tasks. SharePoint 2013 includes four app permission request levels.
The following table lists the four app permission requests that you can assign to a SharePoint 2013 app.
Permission request
Description
Permissions included
Read-Only
Enables apps to view pages, list items, and download documents.
- View Items
- Open Items
- View Versions
- Create Alerts
- Use Self-Service Site Creation
- View Pages
Write
Enables apps to view, add, update, and delete items in existing lists and document libraries.
- Read-Only permissions, plus:
- Add Items
- Edit Items
- Delete Items
- Delete Versions
- Browse Directories
- Edit Personal User Information
- Manage Personal Views
- Add/Remove Personal Web Parts
- Update Personal Web Parts
Manage
Enables apps to view, add, update, delete, approve, and customize items or pages within a web site.
- Write permissions, plus:
- Manage Lists
- Add and Customize Pages
- Apply Themes and Borders
- Apply Style Sheets
Full Control
Enables apps to have full control within the specified scope.
- All permissions
There are important security implications when you assign a permission request level to a SharePoint 2013 app. The permission request level must be adequate to allow the app to function correctly and complete every aspect of the task it is designed to perform. However, it is also important to make sure that the assigned permission request level does not exceed the minimum requirements to complete the task. For example, the Read-Only app permission request level is adequate for a SharePoint 2013 app that is gathering and rendering data in response to a query. However, the Write app permission request level will be required for a SharePoint 2013 app that is intended to add new data or update existing data in a SharePoint 2013 library.
- View Items
-
App authorization policies
In addition to determining the app permission request scope and the app permission request level for each app you deploy, you must also determine which app authorization policy is appropriate. SharePoint 2013 provides the following app authorization policies:
- User and app policy When you assign the user and app policy to a SharePoint 2013 app, the content database authorization checks succeed only if both the current user and the app have sufficient permissions to perform the actions that the app is designed to perform. The user and app policy is required when a SharePoint 2013 site has an embedded IFRAME that links to a SharePoint Store app, and the app calls back to SharePoint 2013 to access SharePoint 2013 resources on behalf of the user. For example, this is a requirement when a SharePoint Store app that does not run within SharePoint 2013 needs to act on behalf of a user to access the user’s resources.
- App-only policy When you assign the app-only policy to a SharePoint 2013 app, the content database authorization checks succeed if the app has sufficient permissions to perform the actions that the app is designed to perform, whether or not the current user (if there is a current user) has the same permissions. The app-only policy is required when the app is not acting on behalf of a user.
- User-only policy When you assign the user-only policy, the content database authorization checks succeed if the user has sufficient permissions to perform the action that the app is designed to perform. The user-only policy is required when a user is accessing their own resources.
- User and app policy When you assign the user and app policy to a SharePoint 2013 app, the content database authorization checks succeed only if both the current user and the app have sufficient permissions to perform the actions that the app is designed to perform. The user and app policy is required when a SharePoint 2013 site has an embedded IFRAME that links to a SharePoint Store app, and the app calls back to SharePoint 2013 to access SharePoint 2013 resources on behalf of the user. For example, this is a requirement when a SharePoint Store app that does not run within SharePoint 2013 needs to act on behalf of a user to access the user’s resources.
-
Configure an environment for apps for SharePoint 2013
Updated: October 16, 2012
Summary: Configure domain names, service applications, and URLs for apps for SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
To enable users to install and use apps for SharePoint in their sites, you must configure your environment to support them. This article describes how to configure your environment to support apps. Use the Plan for apps for SharePoint 2013 article to review options and determine the values to use for configuration settings in this article.
Important:
The steps in this article apply to both SharePoint Foundation 2013 and SharePoint Server 2013.
The following illustration summarizes the steps to take to configure an environment for apps for SharePoint.
Overview of how to configure an environment for apps for SharePoint
These configuration steps result in example app URLs such as the following:
- http://Apps-12345678ABCDEF.ContosoApps.com/sites/SiteName/App1Name/Pages/Home.aspx
- https://Apps-3456789BCDEFG.ContosoApps.com/sites/SiteName/WebName/App2Name/Default.aspx
This article contains instructions for completing these steps.
- http://Apps-12345678ABCDEF.ContosoApps.com/sites/SiteName/App1Name/Pages/Home.aspx
-
Before you begin
- You must purchase a domain name from a domain name provider for your apps, for example, ContosoApps.com.
- You must be a member of the Farm Administrators group to perform the steps in this article. For some steps, you must also be a local administrator on the domain controller.
-
Confirm that the SharePoint Administration (spadmin) and SharePoint Timer (sptimer) services are running.
To verify this, click Start, point to Administrative Tools, and then click Services. In the Services list, verify that the SharePoint Administration and SharePoint Timer services are running.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- You must purchase a domain name from a domain name provider for your apps, for example, ContosoApps.com.
-
Configure the domain names in DNS (all hosting options)
You must configure a new name in Domain Name Services (DNS) to host the apps. To help improve security, the domain name should not be a subdomain of the domain that hosts the SharePoint sites. For example, if the SharePoint sites are at Contoso.com, consider ContosoApps.com instead of App.Contoso.com as the domain name. For more information, see Plan for apps for SharePoint 2013. When an app is provisioned, it provisions a unique DNS domain name (for example, Apps-12345678ABCDEF.ContosoApps.com, where 12345678ABCDEF is a unique identifier for the app). You need a wildcard Canonical Name (CNAME) entry for your DNS domain to support these unique names.
Depending on your configuration (for example, if you are using WINS forward lookup), you might have to create a new forward lookup zone first, or you can start with a wildcard CNAME entry in the same zone as the SharePoint site domain. In the following procedures, you create a forward lookup zone, and then create a wildcard alias record for the DNS domain name that allows for individual apps to create unique domain names within your app domain. In these procedures, we use DNS Manager for Windows Server 2008 R2. For more information about DNS server in Windows Server 2008 R2, see DNS Server. If you have a different type of DNS server, follow the procedures in the documentation for that server type.
To create a forward lookup zone for the app domain name
- Verify that the user account that performs this procedure is a local administrator on the domain controller.
- Click Start, point to Administrative Tools, and then click DNS.
- In DNS Manager, right-click Forward Lookup Zones, and then click New Zone….
- In the New Zone Wizard, click Next.
- In the Zone Type page, accept the default of Primary zone, and then click Next.
- In the Active Directory Zone Replication Scope page, select the appropriate replication method for your environment (the default is To all DNS servers in this domain), and then click Next.
-
In the Zone Name page, in the Zone name box type the name for your new app domain name (for example, ContosoApps.com), and then click Next.
The New Zone Wizard shows the new domain name for apps.
- On the Dynamic Update page, select the appropriate type of dynamic updates for your environment (the default is Do not allow dynamic updates), and then click Next.
- On the Completing the New Zone Wizard page, review the settings, and then click Finish.
For more information about how to create a forward lookup zone, see Add a Forward Lookup Zone.
You have now created a forward lookup zone (and a domain name) to use for apps in your environment.
To create a wildcard Alias (CNAME) record for the new domain name
- Verify that the user account that performs this procedure is a local administrator on the domain controller.
- In DNS Manager, under Forward Lookup Zones, right-click the new app domain name, and then click New Alias (CNAME).
-
In the New Resource Record dialog box, in the Alias name (uses parent domain if left blank) box, type *.
The Fully qualified domain name (FQDN) box displays *. followed by the domain name that you created for apps. For example, *.ContosoApps.com or *.Contoso-Apps.com.
-
Next to the Fully qualified domain name (FQDN) for target host box, type the FQDN of the server that hosts the SharePoint sites.
For example, SharePoint.Contoso.com.
Or:
- Next to the Fully qualified domain name (FQDN) for target host box, click Browse and navigate to the Forward Lookup Zone for the domain that hosts the SharePoint sites.
For example, Contoso.com.
- And then navigate to the record that points to the server that hosts the SharePoint site.
For example, SharePoint.
New Resource Record dialog box shows the wildcard alias for the app domain and the FQDN of the server that hosts the SharePoint sites.
- Next to the Fully qualified domain name (FQDN) for target host box, click Browse and navigate to the Forward Lookup Zone for the domain that hosts the SharePoint sites.
- Click OK.
For more information about how to create a wildcard alias record in DNS Manager, see Add an Alias (CNAME) Resource Record to a Zone.
You can verify the new domain name and alias by pinging them.
To verify the new domain name
- Verify that the user account that is performing this procedure is a local administrator on the domain controller.
- Click Start, and then click Command Prompt.
-
At the command prompt, type ping followed by a subdomain of the domain that you created, and then press ENTER.
For example, ping Apps-12345678ABCDEF.contosoapps.com
If the ping command returns the correct IP address, then your wildcard for the domain name was configured successfully.
-
Create a new wildcard SSL certificate
If you are using Secure Sockets Layer (SSL) for the SharePoint sites in your environment, or if you use any apps that use data external to the SharePoint sites, you should use SSL for your apps. To use SSL, you create an SSL certificate for your app domain (for example, ContosoApps.com).
The domain should be added in the form of a wildcard (for example, *.ContosoApps.com). You need a wildcard certificate instead of individual certificates because each installed app has its own subdomain.
-
Configure the Subscription Settings and App Management service applications
Apps rely on the App Management and Microsoft SharePoint Foundation Subscription Settings service applications. These service applications use the multi-tenancy features to provide app permissions and create the subdomains for apps. Therefore, even if you are not hosting multiple tenants, you must still establish a name for the default tenant for your environment (any SharePoint site that is not associated with a tenant will be in the default tenant).
Note:
You can use the SharePoint Central Administration website to set the default tenant name (also know as the app prefix) for non-hosting environments. You must use Windows PowerShell to configure tenant names for hosting environments. You perform the steps to set the app prefix in the next section, Configure the app URLs to use.
To configure these services, you first start the services in Central Administration. After the services are started, you use Windows PowerShell to create the Subscription Settings service application, and then use either Windows PowerShell or Central Administration to create the App Management service application.
To start the Subscription Settings and App Management services in Central Administration
- Verify that you are a member of the farm administrators group in Central Administration.
- In SharePoint 2013 Central Administration, click System Settings.
- On the System Settings page, under Servers, click Manage services on server.
- On the Services on Server page, next to App Management Service, click Start.
- On the Services on Server page, next to Microsoft SharePoint Foundation Subscription Settings Service, click Start.
-
Verify that the App Management and Microsoft SharePoint Foundation Subscription Settings services are running. The following illustration shows the Services on Server page where you can verify that the App Management and Subscription Settings services are running.
Services on Server showing the App Management and Subscription Settings services running.
To configure the Subscription Settings service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
First you must establish the application pool, run as account, and database settings for the services. Use a farm account for the SPManagedAccount (which will be used for the application pool runas account).
At the Windows PowerShell command prompt, type the following commands, and press ENTER after each one to create the application pool:
$account = Get-SPManagedAccount “<farm account>”
# Gets the name of the Farm administrators account and sets it to the variable $account for later use.
Where:
-
<farm account> is the name of the Farm administrators account in the SharePoint farm.
$appPoolSubSvc = New-SPServiceApplicationPool -Name SettingsServiceAppPool -Account $account
# Creates an application pool for the Subscription Settings service application.
# Uses the Farm administrators account as the security account for the application pool.
# Stores the application pool as a variable for later use.
-
At the Windows PowerShell command prompt, type the following commands, and press ENTER after each one to create the new service application and proxy:
$appSubSvc = New-SPSubscriptionSettingsServiceApplication –ApplicationPool $appPoolSubSvc –Name SettingsServiceApp –DatabaseName <SettingsServiceDB>
# Creates the Subscription Settings service application, using the variable to associate it with the application pool that was created earlier.
# Stores the new service application as a variable for later use.
Where:
-
<SettingsServiceDB> is the name of the Subscription Settings service database.
$proxySubSvc = New-SPSubscriptionSettingsServiceApplicationProxy –ServiceApplication $appSubSvc
# Creates a proxy for the Subscription Settings service application.
For more information, see Get-SPManagedAccount, New-SPServiceApplicationPool, New-SPSubscriptionSettingsServiceApplication, New-SPSubscriptionSettingsServiceApplicationProxy.
You can use either Windows PowerShell or Central Administration to create and configure the App Management service application. The following procedures provide the steps for each method.
To configure the App Management service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
First you must establish the application pool, run as account, and database settings for the services. Use a farm account for the SPManagedAccount (which will be used for the application pool runas account).
At the Windows PowerShell command prompt, type the following commands, and press ENTER after each one to create the application pool:
$account = Get-SPManagedAccount “<farm account>”
# Gets the name of the Farm administrators account and sets it to the variable $account for later use.
Where:
-
<farm account> is the name of the Farm administrators account in the SharePoint farm.
$appPoolAppSvc = New-SPServiceApplicationPool -Name AppServiceAppPool -Account $account
# Creates an application pool for the Application Management service application.
# Uses the Farm administrators account as the security account for the application pool.
# Stores the application pool as a variable for later use.
-
At the Windows PowerShell command prompt, type the following commands, and press ENTER after each one to create the new service application and proxy:
$appAppSvc = New-SPAppManagementServiceApplication -ApplicationPool $appPoolAppSvc -Name AppServiceApp -DatabaseName <AppServiceDB>
# Creates the Application Management service application, using the variable to associate it with the application pool that was created earlier.
# Stores the new service application as a variable for later use.
Where:
-
<AppServiceDB> is the name of the App Management service database.
$proxyAppSvc = New-SPAppManagementServiceApplicationProxy -ServiceApplication $appAppSvc
# Creates a proxy for the Application Management service application.
For more information, see Get-SPManagedAccount, New-SPServiceApplicationPool, New-SPAppManagementServiceApplication and New-SPAppManagementServiceApplicationProxy.
To create the App Management service application in Central Administration
- In SharePoint 2013 Central Administration, on the Application Management page, click Manage service applications.
- On the ribbon, click New, and then click App Management Service.
- In the New App Management Service Application page, in the Service Application Name box, type the name for the service application.
- In the Database section, in the Database Server box, type the instance of SQL Server where you want to store the database, or use the default server.
-
In the Database Name box, type a database name, or use the default name.
The database name must be unique.
- Under Database authentication, select the authentication that you want to use by doing one of the following:
- If you want to use Windows authentication, leave this option selected. We recommend this option because Windows authentication automatically encrypts the password when it connects to SQL Server.
-
If you want to use SQL authentication, click SQL authentication. In the Account box, type the name of the account that you want the service application to use to authenticate to the SQL Server database, and then type the password in the Password box.
Note:
In SQL authentication, an unencrypted password is sent to SQL Server. We recommend that you use SQL authentication only if you force protocol encryption to SQL Server or encrypt network traffic by using IPsec.
- In the Failover Database Server section, if you want to use a failover database server, specify the server name.
- In the Application Pool section, do one of the following:
- Click Use existing application pool, and then select the application pool that you want to use from the drop-down list.
-
Click Create a new application pool, type the name of the new application pool, and then under Select a security account for this application pool do one of the following:
- Click Predefined to use a predefined security account, and then select the security account from the drop-down list.
- Click Configurable to specify a new security account to be used for an existing application pool. You can create a new account by clicking the Register new managed account link.
- Click Predefined to use a predefined security account, and then select the security account from the drop-down list.
- In the Create App Management Service Application Proxy section, leave the Create App Management Service Application Proxy and add it to the default proxy group check box selected.
-
Click OK.
The following illustration shows the App Management service application and proxy that were created.
Manage Service Applications page showing the App Management service application and proxy.
Now you must start the service on the server.
- In SharePoint 2013 Central Administration, click System Settings.
- On the System Settings page, under Servers, click Manage services on server.
- On the Services on Server page, next to App Management Service, click Start.
-
Configure the app URLs to use
In this section, you create the app domain prefix and the tenant name to use for apps in your environment. The app URL points to your app domain and a prefix that determines how each app is named. If you host multiple tenants in your environment, you must use Windows PowerShell to configure the app URLs.
Use the following procedure to configure app URLs for non-hosting (single tenant) environments by using Central Administration.
To configure app URLs
- In Central Administration, click Apps.
- On the Apps page, click Configure App URLs.
-
In the App domain box, type the isolated domain that you created for hosting apps.
For example, ContosoApps.com or Contoso-Apps.com.
-
In the App prefix box, type a name to use for the URL prefix for apps.
For example, you could use “apps“ as the prefix so that you would see a URL for each app such as “apps-12345678ABCDEF.ContosoApps.com“. The following illustration shows the Configure App URLs page after you have filled in the App domain and prefix.
The Configure App URLs page in Central Administration shows the App domain and App prefix.
- Click OK.
Use the following procedure to configure app URLs for multi-tenant hosting environments.
To configure app URLs by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands and press ENTER after each one:
Set-SPAppDomain <appDomain>
Set-SPAppSiteSubscriptionName -Name “app” -Confirm:$false
Where:
- <appDomain> is the domain name that you created.
For more information, see Set-SPAppSiteSubscriptionName and Set-SPAppDomain.
-
Configure the Internet-facing endpoints feature (Optional)
The SharePoint Store contains apps for SharePoint intended for use with sites that require Internet-facing endpoints. By default, these apps are not available (greyed out and cannot be purchased) because they are incompatible with most sites. However, if you have a site that uses Internet-facing endpoints, and want to be able to use these apps, you can turn on the Internet-facing endpoints feature to show these apps in the SharePoint Store. You turn this feature on at the web application level in Central Administration.
To configure Internet-facing endpoints for apps
- In Central Administration, click Application Management.
- On the Application Management page, click Manage Web applications.
- On the Manage Web Applications page, select the web application that you want to change.
- On the Ribbon, click Manage Features.
- In the feature list, next to Apps that require accessible internet facing endpoints, click Activate.
- Click OK.
-
Manage the App Catalog in SharePoint 2013
Published: July 16, 2012
Summary: You can configure and manage an App Catalog for SharePoint environments to control access to available apps.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
You can store apps for SharePoint and Office apps for your organization‘s internal use in an App Catalog site. This article contains an overview of the App Catalog site and shows how to configure the App Catalog for a web application.
In this article:
-
Before you begin
Before you begin:
- Before you configure and use the App Catalog, a member of the Farm Administrators group must configure the environment to support apps for SharePoint. For more information, see Configure an environment for apps for SharePoint 2013.
- You must be a member of the Farm Administrators group to perform the steps in this article.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Before you configure and use the App Catalog, a member of the Farm Administrators group must configure the environment to support apps for SharePoint. For more information, see Configure an environment for apps for SharePoint 2013.
-
Configure the App Catalog site for a web application
The App Catalog site is a special site collection on a web application. Because each web application can have an App Catalog site, a farm can have more than one App Catalog site.
When you create an App Catalog site, you get two libraries for apps:
- Apps for SharePoint
- Apps for Office
Because an App Catalog is scoped to a web application, all apps that you want to make available for a web application have to be in the App Catalog site collection for that web application. You create the App Catalog site collection from SharePoint Central Administration.
To create an App Catalog site collection for a web application
- Apps for SharePoint
- Verify that the user account that is performing this procedure is a member of the Farm administrators group.
-
In Central Administration, on the Apps page, in the App Management section, click Manage App Catalog.
If no App Catalog exists for the farm, the Web Application page opens, so you can select a web application.
- On the Web Application page, select the web application for which you want to create a catalog.
- In the App Catalog Site section, select Create a new app catalog site, and then click OK.
- On the Create App Catalog page, in the Title box, type a title for the App Catalog site.
- In the Description box, type the description for the site.
- In the URL box, fill in the URL to use for the site.
-
In the Primary Site Collection Administrator section, in the User Name box, type the user who will manage the catalog.
Only one user name can be entered. Security groups are not allowed.
-
In the End Users section, in the Users/Groups box, type the names of the users or groups that you want to be able to browse the catalog.
Added users or groups have read access to the App Catalog site. You can add multiple user names and security groups. Users must be added as End Users to be able to browse the App Catalog from their site collections.
- In the Select a quota template list box, select the quota template to use for the site.
- Click OK.
To use an existing App Catalog site collection for a different web application
- Verify that the user account that is performing this procedure is a member of the Farm administrators group.
- In Central Administration, on the Apps page, in the App Management section, click Manage App Catalog.
- On the Manage App Catalog page, next to Web Application, click the down arrow and click Change Web Application.
- In the Select Web Application box, select the web application for which you want to create a catalog.
- In the App Catalog section, select Enter a URL for an existing app catalog site.
- In the URL box, type the URL to the App Catalog site, and then click OK.
To view an App Catalog site collection from Central Administration
- Verify that the user account that is performing this procedure is a member of the Farm administrators group and has Read permission to the App Catalog site.
- In Central Administration, on the Apps page, in the App Management section, click Manage App Catalog.
-
On the Manage App Catalog page, verify that the web application that is selected is the web application you want to manage.
If you want to switch to a different web application, click the down arrow next to the Web application URL to change to a different web application.
- Under Site URL click the link to open the App Catalog for that web application.
-
Configure app requests and SharePoint Store settings
Farm administrators can determine whether users can purchase apps from the SharePoint Store. This setting is at the web application scope. If users cannot purchase apps, they can still browse the SharePoint Store, and request an app. Farm administrators and the App Catalog site owner can view and respond to app requests.
To configure SharePoint Store settings
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group.
- In Central Administration, on the Apps page, in the SharePoint and Office Store section, click Configure Store Settings.
-
On the SharePoint Store Settings page, verify that the selected web application is the web application that you want to configure.
If you want to switch to a different web application, click the down arrow next to the web application URL to change to a different web application.
- To allow or prevent purchases, select an option for Should end users be able to get apps from the SharePoint Store?
- Select Yes to allow users to purchase apps.
- Select No to prevent purchases but allow users to request apps.
- To allow or prevent apps for Office from the Office Store to be started when a user opens a document in the browser, select an option for Should apps for Office from the store be able to start when documents are opened in the browser?
- Select Yes to allow apps for Office from the Office Store to start.
- Select No to prevent apps for Office from the Office Store from starting.
- Click OK.
When users request an app for SharePoint from the SharePoint Store, users can request a specific number of licenses and provide a justification for the purchase of the app for SharePoint. Submitted requests are added to the App Requests list in the App Catalog of the web application that contains a user‘s site collection. The app request includes the following fields:
- Requested by The user name of the person requesting the app for SharePoint.
- Title The title of the app for SharePoint.
- Seats and Site License The number of licenses the user requested for that app for SharePoint.
- Justification The reason why the app for SharePoint would be useful for the organization.
- Status By default, the status is set to New for new requests. The person who reviews the request can change the status to Pending, Approved, Declined, Withdrawn, Closed as Approved, or Closed as Declined.
- View App Details A link to the app details page in the SharePoint Store.
- Approver Comments The person who reviews the request can add comments for the requestor.
To view and manage app requests from the SharePoint Store Settings page
- Verify that the user account that is performing this procedure is a member of the Farm Administrators group and is a member of the site Owners or Designers group for the App Catalog.
- In Central Administration, on the Apps page, in the SharePoint and Office Store section, click Configure Store Settings.
-
On the SharePoint Store Settings page, verify that the selected web application is the web application that you want to configure.
If you want to switch to a different web application, click the down arrow next to the web application URL to change to a different web application.
-
In the App Requests section, click Click here to view app requests.
The App Requests list in the App Catalog site opens.
- Select a request in the list, and then click the Edit button.
- Review the details of the request.
Note:
At this time, the View app details link in the request details opens the SharePoint Store home page, instead of the details page for the app. Search for the app in the SharePoint Store to find more information about the app.
- Change the Status to the appropriate value – Approved if you want to user to be able to purchase the app, or Declined if you do not want to allow the purchase.
-
Add comments in the Approver Comments box, and then click Save.
To view a request, requestors can go to the Add an App page in their site collection, and then click Your Requests.
To view and manage app requests from the App Catalog site
- Verify that the user account that is performing this procedure is a member of the site Owners or Designers group for the App Catalog.
- On the App Catalog site, click the App Requests list.
- Select a request in the list, and then click the Edit button.
- Review the details of the request.
Note:
At this time, the View app details link in the request details opens the SharePoint Store home page, instead of the details page for the app. Search for the app in the SharePoint Store to find more information about the app.
- Change the Status to the appropriate value – Approved if you want to user to be able to purchase the app, or Declined if you do not want to allow the app to be purchased.
-
Add any comments in the Approver Comments box, and then click Save.
To view a request, requestors can go to the Add an App page in their site collection, and then click Your Requests.
-
Add apps to the App Catalog
After you have configured the App Catalog, you can add apps that users can then install to their SharePoint sites or use in their Office documents.
To add an app to the App Catalog
- Verify that the user account that is performing this procedure is a member of the site Owners or Designers group for the App Catalog.
-
On the App Catalog site, click the Apps for SharePoint list.
On the Apps for SharePoint page, click new item.
- In the Choose a file box, click Browse, and then locate the folder that contains the app that you want to upload.
Tip:
You can also click Upload files using Windows Explorer instead to drag and drop an app for SharePoint into the App Catalog.
- Select the app, and then click Open.
- Click OK to upload the app.
-
In the Item details box, verify the Name, Title, Short Description, Icon URL, and other settings for the app.
Be sure that the Enabled check box is selected so that users can see the app in their sites.
You can select the Featured check box to list the app in the Featured content view of the App Catalog.
- Click Save.
You can also categorize apps in the App Catalog. To add categories, edit the Category field for the App Catalog list and add the category names you want to use.
You can preview how the app will appear to users.
-
Remove apps from the App Catalog
If you no longer want to offer a particular app to your users, you can remove it from the App Catalog. Removal does not uninstall or remove the app from sites to which it has been added. It merely removes the app from the App Catalog, and users cannot add the app to other sites.
To remove an app from the App Catalog
- Verify that the user account that is performing this procedure is a member of the site Owners or Designers group for the App Catalog.
- On the App Catalog site, click the Apps for SharePoint list.
- On the Apps for SharePoint page, select the app that you want to remove.
- In the ribbon, on the Files tab, click Delete Document to remove the app.
-
In the dialog box, click OK to confirm that you want to send the item to the site Recycle Bin.
The app is removed.
-
Add apps for SharePoint to a SharePoint 2013 site
Published: July 16, 2012
Summary: Site owners can add apps for SharePoint to SharePoint sites so that they and other users of the site can use the app.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
Site owners can add apps for SharePoint from the SharePoint Store or an App Catalog to their sites. Adding an app installs an instance of that app to the site. In addition, several lists, libraries, and other SharePoint components, which are also called apps in SharePoint 2013, are available to add to a site.
-
Before you begin
Before you begin this operation, review the following information about prerequisites and permissions:
- Before a user can add an app for SharePoint, a member of the Farm Administrators group must configure the environment to support apps for SharePoint. For more information, see Configure an environment for apps for SharePoint 2013.
- A user must have the Manage Web site and Create Subsites permissions to add an app for SharePoint. By default, these permissions are available only to users who have the Full Control permission level or who are in the site Owners group.
- When a user adds an app for SharePoint, the app requests permissions that it needs to function (for example, access to Search, or to create a list). Users who do not have those permissions are informed that they do not have sufficient permissions and the app cannot be added. The user can contact a site or farm administrator to see if the administrator can add the app.
- A user logged in to a site as the system account cannot install an app. The system account cannot import app licenses because that could result in performance problems.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Before a user can add an app for SharePoint, a member of the Farm Administrators group must configure the environment to support apps for SharePoint. For more information, see Configure an environment for apps for SharePoint 2013.
-
Add apps for SharePoint to SharePoint sites
Site owners can add apps for SharePoint from the following sources to their sites:
- from the list of apps already available for a site (default apps, such as standard lists and libraries, and apps that have been purchased already).
- from the App Catalog.
- from the SharePoint Store.
The following procedures provide steps for adding apps from these sources.
To add an app from the list of available apps in a site
- from the list of apps already available for a site (default apps, such as standard lists and libraries, and apps that have been purchased already).
- Verify that the user account that is performing this procedure is a member of the site Owners group.
-
On the home page, under Get started with your site, click Add lists, libraries, and other apps.
If the Get started with your site control does not appear on the home page, click the Settings icon, and click View Site Contents, and then on the Site Contents page, click Add an App.
- In the Your Apps list, click the app you want to add.
-
Follow the instructions to Trust the app (if it is a custom component) or Name the app (if it is a SharePoint component).
The app for SharePoint is added and appears in the Apps section of your Site Contents list.
To add an app from an App Catalog
- Verify that the user account that is performing this procedure is a member of the site Owners group.
-
On the home page, under Get started with your site, click Add lists, libraries, and other apps.
If the Get started with your site control does not appear on the home page, click the Settings icon, and click View Site Contents, and then on the Site Contents page, click Add an App.
-
Click FromName.
Where Name is the name of your organization’s App Catalog. For example, “From Contoso”.
Tip:
Apps marked as Featured in the App Catalog will also appear in the main list of Apps.
- Click the app you want to add.
-
In the Grant Permission to an App dialog box, if you trust the app, click Allow Access.
The app for SharePoint is added and appears in Apps section of your Site Contents list.
To add an app from the SharePoint Store
- Verify that the user account that is performing this procedure is a member of the site Owners group.
-
On the home page, under Get started with your site, click Add lists, libraries, and other apps.
If the Get started with your site control does not appear on the home page, click the Settings icon, and click View Site Contents, and then on the Site Contents page, click Add an App.
- Click SharePoint Store.
- Browse the SharePoint Store to find an app that you want.
- Click the app you want to add.
- Click Details, and then click Buy It.
- Follow the steps to log in and purchase the app, if required.
-
In the Grant Permission to an App dialog box, if you trust the app, click Allow Access.
The app for SharePoint is added and appears in the Apps section of your Site Contents list.
You can also install an app by using Windows PowerShell. First, you import the app package from the file system, and then install it to the site collection. The following procedure contains a script to perform these steps.
To install an app by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Site Owners group on the site collection to which you want to install the app.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command to import the app and then press ENTER:
$spapp = Import-SPAppPackage -Path Path to app -Site URL -Source Source
# Imports the app and sets a variable that you can use to identify the app when you install it in the next step.
Where:
- Path to app is the path to the app you want to import on the file system.
- URL is URL for the site collection to which you want to import the app.
- Source is one of the following: Marketplace, CorporateCatalog, DeveloperSite, ObjectModel, RemoteObjectModel, or InvalidSource.
-
At the question Are you sure you want to perform this action?, type Y to import the app.
The app is imported and information about the app, including the Asset ID, version string, and Product ID is displayed.
-
At the Windows PowerShell command prompt, type the following command to add the app to a site and then press ENTER:
Install-SPApp -Web URL -Identity $spapp
# Installs the app to the subweb you specify.
# Uses the $spapp variable you set previously to identify that app you want to install.
Where:
- URL is URL for the site or subweb to which you want to install the app.
For more information, see Import-SPAppPackage and Install-SPApp.
-
Remove an app for SharePoint from a SharePoint 2013 site
Published: July 16, 2012
Summary: When administrators remove apps for SharePoint from SharePoint sites, the apps are uninstalled and functionality is no longer available to users.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
In this article:
-
Before you begin
Before you begin this operation, review the following information about permissions:
- A user must have the Manage Web site permission to remove an app for SharePoint. By default, this permission is only available to users with the Full Control permission level or who are in the site Owners group.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support (http://go.microsoft.com/fwlink/p/?LinkId=246502)
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products (http://go.microsoft.com/fwlink/p/?LinkId=246501)
- Keyboard shortcuts (http://go.microsoft.com/fwlink/p/?LinkID=246504)
- Touch (http://go.microsoft.com/fwlink/p/?LinkId=246506)
- A user must have the Manage Web site permission to remove an app for SharePoint. By default, this permission is only available to users with the Full Control permission level or who are in the site Owners group.
-
Remove an app from a SharePoint site
Site owners can remove apps for SharePoint from their sites. The following procedures provide steps for removing (or uninstalling) an app. When you remove an app, the data for that app will no longer be available.
To remove an app from a SharePoint site
- Verify that the user account that is performing this procedure is a member of the Site owners group.
- On the site, on the Settings menu, click View Site Contents.
- In the Apps section, point to the app that you want to remove, click …, and then click Remove.
- Click OK to confirm that you want to remove the app.
Before you use the following procedure, be sure to get the title for the app that you want to remove.
To remove an app by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Site Owners group on the site collection to which you want to install the app.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands, and press ENTER after each one:
$instances = Get-SPAppInstance -Web <URL>
# Gets all apps installed to the subsite you specify.
$instance = $instances | where {$_.Title -eq ‘<App_Title>‘}
# Sets the $instance variable to the app with the title you supply.
Uninstall-SPAppInstance -Identity $instance
# Uninstalls the app from the subsite.
Where:
- <URL> is the path site collection or subsite that contains the app.
- <App_Title> is the title of the app you want to remove.
- At the question Are you sure you want to perform this action?, type Y to uninstall the app.
For more information, see Get-SPAppInstance, Uninstall-SPAppInstance.
-
Monitor apps for SharePoint for SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how administrators and site owners can monitor the health and usage details for apps for SharePoint in SharePoint Server 2013.
Applies to: SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
You can use the SharePoint Central Administration website to add and remove apps for SharePoint and check details and errors.
Important:
- The steps in this article apply to SharePoint Server 2013 only.
- App monitoring is not available in SharePoint Foundation 2013.
- App monitoring is not supported for SharePoint Server 2013 on-premise, multi-tenancy environments.
In this article:
- The steps in this article apply to SharePoint Server 2013 only.
-
Before you begin
Before you can monitor apps, a member of the Farm Administrators group must configure the environment to support apps for SharePoint. For more information, see Configure an environment for apps for SharePoint 2013.
You must be a member of the Farm Administrators group or the site Owners group to perform the steps in this article.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Selecting apps to monitor in Central Administration
The Monitored Apps page displays the apps for SharePoint that a Farm Administrator monitors. Each app for SharePoint that is listed on this page includes details to help an administrator monitor performance. For example, each app for SharePoint provides the following properties: Name, Status, Source, Licenses in Use, Licenses Purchased, Install Locations, and Runtime Errors. A Farm Administrator chooses to add, remove, and monitor apps for SharePoint.
Important:
The Monitor Apps page requires the following search analytics and usage file import timer jobs to be active:
- ECM analytics timer job name: Usage Analytics timer job for Search Service
- Usage DB timer job name: Microsoft SharePoint Foundation Usage Data Import
For more information, see Timer job reference (SharePoint 2013)
- ECM analytics timer job name: Usage Analytics timer job for Search Service
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- In Central Administration, click General Application Settings.
- On the General Application Settings page, in the Apps section, click Monitor Apps.
- On the Monitored Apps page, in the Action group of the ribbon, click Add App.
Note:
If the App Catalog is not already created, or if the App Management Service application and app domain settings are not configured correctly the Add App dialog may create an error.
- Select the checkbox for the app that you want to monitor, or type a name in the Search for app name box, and then click the Search icon.
- On the search results page, select the app that you want to monitor.
Note:
Apps that you add to the Monitored Apps list previously are not displayed in the search results.
-
Click Add App.
The app now appears in the list of monitored apps.
To remove an app from the monitor apps list
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Monitored Apps page, select the checkbox next to the app that you want to remove.
- In the Manage group of the ribbon, click Remove App.
-
Monitoring app details in Central Administration
This section explains how farm administrators can monitor and understand the apps for SharePoint details. There are multiple ways that an administrator can view the error and usage details for apps for SharePoint. By selecting an app in the Monitored Apps page, an administrator can use the ribbon to access the error or usage details for that app. An administrator can also click an app in the list on the Monitored Apps page to open the app details page and access the same error or usage details.
The app usage and app error details data that is in the app monitoring pages can be delayed for up to 29 hours. The app details depend on the when the ECM analytics timer job is scheduled to run. When the timer job runs, it collects events for the previous day. For example, if the timer job is scheduled to run at 5 A.M., then the most recent events that are collected are from 11:59 P.M. the previous day. Zn event that occurs at 12:01 A.M. will not appear in the app details pages until up to 29 hours later.
Note that if you view the app error details page for a specific instance of an app, the number of errors for the app is synchronized with the error messages in the list. This occurs because the number of errors appears in the app error details page instead of the events that are processed by the ECM analytics timer job.
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
-
On the Monitored Apps page, click the app that you want to view.
A new page opens and displays detailed information about the app, such as the following: licensing, errors, installations, and usage.
Note:
The administrator can also select an app in the monitored apps list and in the App Details group of the ribbon, click View Details.
- In the Usage section, click Days, Months, or Years to change the chart to those time frames.
To view the app error details in Monitored Apps
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Monitored Apps page, click the number in the Runtime Errors column for the app you want to view.
Note:
The administrator can also select an app in the monitored apps list and in the App Details group of the ribbon, click View Errors.
- The App Monitoring Details dialog appears with information about each error for that app. You can use the Correlation ID to find the errors in the error log.
- Click the URL in the Location column to view more error details for this app.
- On the App Monitoring Details page, click the number next to Runtime Errors.
- The App Monitoring Details dialog appears and includes a list of all Runtime Errors for this app, the time each error occurred, and the Correlation ID.
Note:
The app error list can help you determine if you want to remove the app because there are too many errors or if the app is working as it should.
-
Monitoring app details in a SharePoint site
This section explains how site owners can monitor and understand the usage of apps for SharePoint. A site owner can view the error and usage details for apps for SharePoint by selecting an app in the Site Contents page and then clicking Monitor in the app dialog box.
Note:
Be aware that in the app details page, the error dialog box can show more errors than are counted. The errors are counted every 24 hours, but the error messages are processed more often. As a result, the error dialog box can show error messages that are generated in the current day before the count is updated at the end of the day.
- Verify that the user account that is performing this procedure is a member of the site Owners group.
-
On the Site Contents page, in the quick launch pane, click Apps.
A new page opens and displays all of the apps that are installed on this site.
-
On the Apps page click the icon next to the app you want to monitor and then click Details in the callout.
The App Details page appears for the selected app and the site owner can see the details for licenses, errors Installs and usage.
-
In the Errors section, click the number next to Install Errors, Runtime Errors, or Upgrade Errors to see the error details.
For example, click the number next to Runtime Errors and the Runtime Errors dialog appears. This includes a list of all Runtime Errors for this app, the time each error occurred, and the Correlation ID.
This app error list can help you determine if you want to remove the app because there are too many errors or if the app is working as it should.
Note:
The app errors that appear in this list have occurred within the previous four days.
-
In the Usage section, click Days, Months, or Years to change the chart to those time frames.
The chart displays two bars for each time period that represents the number of times the app has been launched and the number of specific users that use this app each day.
Note:
If the app uses connections to external data sources through Business Connectivity Services, a graph that shows the number of calls made to the external data sources is also shown. Dates that appear in the Usage and BCS Calls graphs are in Coordinated Universal Time (UTC).
-
Monitor and manage app licenses in SharePoint Server 2013
Published: July 16, 2012
Summary: Learn how SharePoint farm administrators assign, monitor, and manage the app for SharePoint licenses in SharePoint Server 2013.
Applies to: SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
You can use the SharePoint Central Administration website to monitor and manage licenses for the apps for SharePoint. Licenses for apps for SharePoint are digital sets of verifiable information that state the user rights for a app for SharePoint. All apps that are distributed through the SharePoint Store are the only apps that have built-in licenses that SharePoint Server 2013 recognizes.
Members of the Farm Administrators group manage licenses for apps and can also assign license managers for others to manage app for SharePoint licenses.
Important:
The steps in this article apply to SharePoint Server 2013 only.
-
Before you begin
Before you can monitor and manage app licenses you must configure your environment to support apps for SharePoint. You must be a member of the Farm Administrators group to perform these steps. For more information, see Configure an environment for apps for SharePoint 2013.
Important:
We support only one App Management Service Application per farm. This helps ensure that the app license feature functions correctly.
Before you begin this operation, review the following information about what SharePoint Server 2013 does and does not provide for apps for SharePoint licensing:
-
SharePoint Server 2013 provides:
- Storefront to obtain apps
- Storage and renewal of app for SharePoint licenses
- User Interface (UI) to assign users to specific app for SharePoint licenses
- APIs for developers to query for license information
- Storefront to obtain apps
- SharePoint Server 2013 does not enforce app for SharePoint licenses.
- Developers must add code in their apps for SharePoint to retrieve license information and react accordingly.
- All app for SharePoint licenses are bound to a specific SharePoint Server 2013 deployment but can be transferred to a different SharePoint Server 2013 deployment three times.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
-
Monitoring and managing app licenses
A farm administrator or a license manager can check the licenses for all apps for SharePoint on the App Licenses page. It is important to track the number of licenses that are available for each app for SharePoint so that users do not exceed this number. An administrator can assign additional users to a app for SharePoint license, purchase additional licenses for an app, and also add managers to a license. For more information about how to monitor apps for SharePoint see, Monitor apps for SharePoint for SharePoint Server 2013.
To view app license details
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group or a license manager.
- In Central Administration, click Apps.
- On the Apps page, in the Store section, click Manage App Licenses.
-
On the Manage App Licenses page, click an app for SharePoint in the list to view the license details.
The Manage App License page shows detailed licensing information. This includes the name of the app, the developer, and current license details.
-
In the top section, click the drop-down arrow in the dialog box to see purchase details for the selected app for SharePoint.
The app details include the following information:
- Number of licenses available for users
- License type
- App purchaser name
- At the end of the dialog box, a farm administrator can view the app details.
-
Click View in Store to see the app details.
- In the People with a License (number of licenses available)section, the number of available licenses and a list of the people who currently have licenses for this App are shown.
- In the License Managers section, all app managers are listed.
- In the People with a License (number of licenses available)section, the number of available licenses and a list of the people who currently have licenses for this App are shown.
To add users to the app license
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Manage App Licenses page, click an app for SharePoint for which you want to add users.
- In the People with a License section, click assign people.
-
In the dialog box that appears below, enter the user name that you want to add and then, click Add User.
The user name is added to the list at the bottom of this section and the number of available licenses for this app is refreshed for the selected app for SharePoint.
To purchase more app licenses
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Manage App Licenses page, click an app for SharePoint for which you want to purchase more licenses.
- In the People with a License section, click buy more licenses.
- The SharePoint Store opens with the specific app showing the details with links to purchase additional licenses. Choose the number of Apps you want to purchase and then click OK.
To remove app licenses
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Manage App Licenses page, click an app for SharePoint for which you want to remove licenses.
- In the top section, under the app for SharePoint name, at the end of the dialog box, click Remove this License.
- Verification: Optionally, include steps that users should perform to verify that the operation was successful.
To recover app licenses
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
- On the Manage App Licenses page, click an app for SharePoint for which you want to recover licenses.
-
In the top section, under the app name, at the end of the dialog box, click Recover License.
The app for SharePoint details show any changes the administrator has made.
To add a license manager
- Verify that the user account that is performing this procedure is a member of the Farm Administrators SharePoint group.
-
On the Manage App License page, in the License Managers section, click add manager.
Below the License Managers section, the new App manager appears in the list.
-
Upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Learn how to plan, prepare, and perform an upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about performing an upgrade to SharePoint 2013.
-
Downloadable resources about upgrade
Download the following content for information about upgrade.
Content
Description
Describes the steps in the process for a database-attach upgrade.
SharePoint 2013 Products Preview – Test Your Upgrade Process model
See a visual display of information about how to test the upgrade process.
Use this worksheet to record information about your environment while you test upgrade.
-
TechNet articles about upgrade
The following articles about upgrade are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Find resources to help you understand how to upgrade databases and site collections from SharePoint 2010 Products to SharePoint 2013.
Find resources about how to plan to upgrade from SharePoint 2010 Products to SharePoint 2013.
Find resources about how to test and troubleshoot an upgrade from SharePoint 2010 Products to SharePoint 2013.
Find resources to help you perform the steps to upgrade databases from SharePoint 2010 Products to SharePoint 2013.
Find out how to upgrade a site collection to SharePoint 2013.
-
Additional resources about upgrade
The following resources about upgrade are available from other subject matter experts.
Content
Description
Upgrade and Migration Resource Center for SharePoint 2013 Products
Visit the Resource Center to find additional information about upgrades to SharePoint 2013.
Capabilities and features in SharePoint 2013 Resource Center
Visit the Resource Center to learn about what‘s new in SharePoint 2013.
-
Get started with upgrades to SharePoint 2013
Published: July 16, 2012
Summary: Find resources to help you understand how to upgrade databases and site collections from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The first step in any upgrade process is to learn about the process itself so that you can plan and prepare appropriately. These articles help you understand how the process of upgrading from SharePoint 2010 Products to SharePoint 2013 works. These articles also include overviews of how to upgrade service applications.
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about understanding upgrade for SharePoint 2013.
-
Downloadable resources about upgrade to SharePoint 2013
Download the following content for information about upgrade.
Content
Description
Describes the steps in the process for a database attach upgrade
-
TechNet articles about understanding upgrade
The following articles about understanding upgrade are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Find out about new requirements, approaches, and features that are available for upgrading to SharePoint 2013.
Get a visual overview of the steps involved in performing an upgrade.
SharePoint 2010 Products included several service applications, some of which have databases that can be upgraded when you upgrade to SharePoint 2013. Find out which service application databases can be upgraded and what steps that you must take before, during, and after upgrade for your service applications.
Upgrade farms that share services (parent and child farms) to SharePoint 2013
In SharePoint Server 2010, it was possible to configure parent farms and child farms to share services. In such an environment, the parent farm hosts one or more service applications from which one or more child farms consume services. Learn how to approach upgrading these environments to SharePoint Server 2013.
Get off to the right start – review these best practices for testing and performing an upgrade to SharePoint 2013.
Review supported editions and products for upgrading to SharePoint 2013
Understand the requirements for upgrade. And if you are planning to change SKUs or products during upgrade, understand which upgrade paths are supported.
-
Additional resources about upgrade to SharePoint 2013
The following resources about upgrade to SharePoint 2013 are available from other subject matter experts.
Content
Description
Upgrade and migration for SharePoint 2013 IT Pros Resource Center
Visit the Resource Center to access videos, community sites, documentation, and more.
-
What’s new in SharePoint 2013 upgrade
Published: July 16, 2012
Summary: SharePoint 2013 includes new upgrade features, such as upgrade for service applications, a site health checker, and upgrade for site collections.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
SharePoint 2013 does not support in-place upgrade for an existing environment. You must use the database-attach upgrade method to upgrade your databases to a new environment that is based on SharePoint 2013. Also, to provide more flexibility to farm administrators and site administrators, the upgrade process has changed to separate upgrade of the software and databases from upgrade of the sites.
-
In-place upgrade of the farm is not supported
An upgrade to SharePoint Server 2010 or SharePoint Foundation 2010 provides an option to install the new version over the earlier version on the same hardware. This is called an in-place upgrade. During this process, the complete installation, that includes databases and sites, is upgraded in a fixed order. Although this is a simple method, in-place upgrade presents problems in performance and control for a farm administrator. There was no way to control the order in which content is upgraded, and a failure in a particular site collection could stop the whole process.
The database-attach upgrade method offers more flexibility, more control, and a better success rate. To use a database attach upgrade, you complete the following tasks:
- Create and configure a new farm that is separate from the old farm
- Copy the content and services databases to the new farm
- Upgrade the data and sites
You can upgrade the content databases in any order and upgrade several databases at the same time to speed up the overall process.
For more information, see Overview of the upgrade process to SharePoint 2013.
-
Database-attach upgrade is available for some service application databases
For the SharePoint 2013, you can use the database attach upgrade method to upgrade the following service application databases:
-
Business Data Connectivity
This service application is available for both SharePoint Server 2013 and SharePoint Foundation 2013.
-
Managed Metadata
This service application is available only for SharePoint Server 2013.
-
PerformancePoint
This service application is available only for SharePoint Server 2013.
-
Secure Store
This service application is available only for SharePoint Server 2013.
-
User Profile (Profile, Social, and Sync databases)
This service application is available only for SharePoint Server 2013.
-
Search administration
This service application is available only for SharePoint Server 2013.
For more information, see Services upgrade overview for SharePoint Server 2013.
-
-
Deferred site collection upgrade
In SharePoint 2010 Products, farm administrators use either the in-place upgrade process to upgrade sites immediately, or the command line to upgrade all sites at the same time or individually. In SharePoint 2013, farm administrators can now allow site collection owners to upgrade their sites to the new user interface on their own timeline. The commands for upgrading a site collection are on the Site Settings page in the Site Collection Administration section. There are also Windows PowerShell cmdlets to upgrade site collections to the new user interface. For more information, see Plan for site collection upgrades in SharePoint 2013, Upgrade a site collection to SharePoint 2013and Manage site collection upgrades to SharePoint 2013.
-
Site collection health checker
Site collection owners or administrators can use a site collection health checker to detect any potential issues with their site collections and address them before they upgrade sites to the new version. The checker is available after upgrade also to detect any health issues on an ongoing basis. Note that some issues can be repaired automatically, but others require manual steps to repair. During a site collection upgrade, if the checker finds issues that can be repaired automatically, they are repaired at that time. For more information, see Run site collection health checks in SharePoint 2013.
-
Upgrade evaluation site collections
In SharePoint 2013, the upgrade of the software and data was separated from the upgrade of the site. This means that the sites can truly remain running in SharePoint 2010 mode until a site owner or administrator explicitly upgrades the site to the new user interface. Site collection owners can request an evaluation site, which is a separate copy of the site, to review the new interface and functionality. After they have reviewed the site and made necessary changes in their original site, they can then upgrade their sites to the new version. Evaluation sites are set to automatically expire and be deleted. For more information, see Plan for upgrade evaluation sites, Create an upgrade evaluation site (Optional), and Manage site collection upgrades to SharePoint 2013.
-
Notifications for life-cycle events
An email message and a status bar notification in a site collection notifies site collection owners when an upgrade is available. Site collection owners can create an evaluation site from email and control the expiration and deletion of that site by using email also. A status bar notification in the site collection also informs all users if a site is in read-only mode. For more information, see Plan settings for upgrade notifications, self-service upgrade, and site collection creation and Manage site collection upgrades to SharePoint 2013.
-
Throttles for site collection upgrade
To make sure that site collection upgrades do not cause an outage on your farm, there are throttles built in at the web application, database, and content level. This means that even if 100 site collection owners decide to upgrade their site collections at the same time, only some are run at the same time, and the rest are put into a queue to run later. For more information, see Plan site collection upgrade throttling and queues and Manage site collection upgrades to SharePoint 2013.
-
True “SharePoint 2010” instead of visual upgrade
Visual upgrade in SharePoint 2010 Products lets site owners and administrators see what their site would be like in the new user interface. However, it is not a true preview because the site itself has already been upgraded to the new functionality. Consequently, some Web Parts or other elements do not display correctly.
SharePoint 2013 can host sites in both SharePoint 2010 and SharePoint 2013 modes. The installation contains both SharePoint 2010 and SharePoint 2013 versions of the following types of elements:
-
Features, site templates, site definitions, and Web Parts
The directories on the file system are duplicated in both the 14 and 15 paths, for example:
- Web Server Extensions/14/TEMPLATE/Features
- Web Server Extensions/15/TEMPLATE/Features
- Web Server Extensions/14/TEMPLATE/Features
-
IIS support directories:
- _Layouts, _Layouts/15
- _ControlTemplates, _ControlTemplates/15
- _Layouts, _Layouts/15
-
Solution deployment, which lets legacy solutions work in 2010 mode
Note that existing SharePoint 2010 Products solutions can be deployed to SharePoint 2013 and continue to function for 2010 sites, usually without requiring any changes.
Because of these directories, you can continue hosting unupgraded sites in an upgraded environment until all site collections are ready to upgrade. For more information, see About site collection modes.
-
-
Log files now in ULS format
The format of the upgrade, upgrade error, and site upgrade log files now comply with the Unified Logging System (ULS) conventions for easier review. For more information, see Verify database upgrades in SharePoint 2013.
-
Overview of the upgrade process to SharePoint 2013
Published: July 16, 2012
Summary: Learn about the process of upgrading databases, service applications, My Sites, and site collections to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
To upgrade from SharePoint 2010 Products to SharePoint 2013, you use the database-attach method to upgrade. In the database-attach method, you first create and configure a SharePoint 2013 farm. Then you copy the content and service application databases from the SharePoint 2010 Products farm, and then attach and upgrade the databases. This upgrades the data to the new version. Site owners can then upgrade individual site collections.
Figure: The sequence of upgrade stages
This article helps you understand the upgrade sequence so that you can plan an upgrade project. To get detailed steps for an upgrade, see Upgrade databases from SharePoint 2010 to SharePoint 2013 and Upgrade site collections to SharePoint 2013.
Important:
This article applies to both SharePoint Foundation 2013 and SharePoint Server 2013, except for information about how to upgrade My Sites and specific service applications that are only in SharePoint Server 2013.
-
Create the SharePoint 2013 farm
The first stage in the upgrade process creates the new SharePoint 2013 farm:
- A server farm administrator installs SharePoint 2013 to a new farm. The administrator configures farm settings and tests the environment.
-
A server farm administrator sets the SharePoint 2010 Products farm to read-only so that users can continue to access the old farm while upgrade is in progress on the new farm.
Figure: Create new farm, set old farm to read-only
-
Copy the SharePoint 2010 Products databases
The second stage in the upgrade process copies the databases to the new environment. You use SQL Server Management Studio for these tasks.
- With the farm and databases in read-only mode, a server farm administrator backs up the content and service application databases from the SQL Server instance on the SharePoint 2010 Products farm.
-
The server farm administrator restores a copy of the databases to the SQL Server instance on the SharePoint 2013 farm and sets the databases to read-write on the new farm.
Figure: Use SQL Server tools to copy databases
-
Upgrade SharePoint 2010 Products databases and service applications
The third stage in the upgrade process upgrades the databases and service applications.
- A server farm administrator configures the service applications for the new farm. The following service applications have databases that you can upgrade during this process:
-
SharePoint Server 2010 and SharePoint Foundation 2010
- Business Data Connectivity service application
- Business Data Connectivity service application
-
SharePoint Server 2010 only
- Managed Metadata service application
- PerformancePoint Services service application
- Search service application
- Secure Store Service application
- User Profile service application
- Managed Metadata service application
-
A server farm administrator creates a web application on the SharePoint 2013 farm for each web application on the SharePoint 2010 Products farm.
Figure: Create web applications for upgrade
-
A server farm administrator installs all server-side customizations.
Figure: Copy customizations to the new farm
-
A server farm administrator then attaches the content databases to the new farm and upgrades the content databases for those web applications.
Figure: Upgrade the databases by using Windows PowerShell
- A server farm administrator confirms that the upgrade is successful.
-
Upgrade SharePoint 2010 Products site collections
The final stage in the upgrade process is to upgrade the site collections. In SharePoint 2013, site owners are in charge of upgrading their sites. The upgrade process for My Sites is slightly different from for other types of site collections.
Important:
This section applies to SharePoint Server 2013 only.
A server farm administrator upgrades the My Site host and then individual users can upgrade their My Sites or the farm administrator can upgrade them by using Windows PowerShell. The following illustration shows four stages for the My Site host and My Sites during the upgrade process.
Figure: Stages in upgrading My Sites
- The My Site host has not been upgraded. My Sites cannot be upgraded yet.
- A server farm administrator has upgraded the My Site host. No My Sites have been upgraded.
- Some users have upgraded their My Sites.
- All My Sites have been upgraded.
Note:
A server farm administrator can choose to force an upgrade of My Sites without waiting for users to upgrade them. For details and steps, read Upgrade site collections to SharePoint 2013.
Owners of all other site collections can start to upgrade their sites as soon as they see a notification on their site’s home page that the new version is available. The following illustration shows four stages for a site collection during the upgrade process.
Stages in upgrading site collections
- The site owner runs the site collection health checks to determine readiness for upgrade. The site owner addresses issues before they continue with the next step.
-
Optionally, the site owner requests an upgrade evaluation site collection. A timer job runs to create the site collection and the site owner receives an email message when the evaluation site collection is ready. The site owner previews the new user interface. After several days or weeks, the evaluation site collection expires and is deleted by a timer job.
A server farm administrator can determine the length of time before expiration.
- When the site owner is ready, the site owner starts the upgrade process. The site collection health checks are run again automatically. The site owner must address issues before upgrading. If health checks return no issues, the upgrade starts.
- When upgrade is complete, the site owner sees the Upgrade Status page that contains the status and a link to the upgrade logs. The site owner reviews the site to make sure that everything works correctly.
Note:
A server farm administrator can also force specific site collections to be upgraded without waiting for the site owners to upgrade them. For details and steps, read Upgrade site collections to SharePoint 2013.
-
Services upgrade overview for SharePoint Server 2013
Published: July 16, 2012
Summary: Create a plan to upgrade data for service applications when you upgrade from SharePoint Server 2010 to SharePoint Server 2013.
Applies to: SharePoint Server 2013
The upgrade process for SharePoint Server 2013 uses the database attach upgrade method. When you move your databases to a new farm and upgrade the content, you must create your services infrastructure in the new farm, and configure the services appropriately for your new farm and new version. The following service applications have databases that can be upgraded when you upgrade from SharePoint Server 2010 to SharePoint Server 2013:
- Business Data Connectivity service application
- Managed Metadata service application
- PerformancePoint Services service application
- Search service application
- Secure Store Service application
- User Profile service application
Attaching and upgrading these databases configures these service applications. Settings for other services will have to be reconfigured when you upgrade.
Important:
The content in this article about the Business Data Connectivity service application applies to both SharePoint Foundation 2013 and SharePoint Server 2013. Other services are available only in SharePoint Server 2013.
- Business Data Connectivity service application
-
Database attach upgrade with services
You must create the service applications on your new farm before you upgrade your content databases. The steps included in the installation guide above describe how to use the Farm Configuration Wizard to enable all service applications. Some service applications can be upgraded by using a service application database upgrade. If you want to upgrade these service applications by upgrading the service application databases, you should not use the Farm Configuration Wizard to configure these service applications when you set up your new farm.
The following service applications can be upgraded by performing a services database upgrade:
-
Business Data Connectivity service
The Business Data Connectivity service uses a database to store information about external data. This database must be upgraded as part of a services database attach upgrade. This service application is also available in SharePoint Foundation 2013.
-
Managed Metadata service
The Managed Metadata service uses a database to store metadata information. This database must be upgraded as part of a services database attach upgrade. You must attach and upgrade the database for this service and for the User Profile service before you can upgrade any My Sites.
-
PerformancePoint services
PerformancePoint Services use a database to store information. This database must be upgraded as part of a services database attach upgrade.
-
Search
In SharePoint Server 2010, the Search service application Administration database contains settings for the Search service application such as content sources, crawl rules, start addresses, server name mapping, and federated locations. You can upgrade a Search service application Administration database from SharePoint Server 2010 to SharePoint Server 2013 by using a database attach approach.
You cannot use the database attach approach to upgrade any other search databases, such as crawl databases or property databases. (These databases are re-created when you perform a full crawl in the new farm.) Also, the upgrade process does not preserve or upgrade logical components of the SharePoint Server 2010 farm topology. After you perform the upgrade, you must manually re-create a topology as appropriate for the requirements of the organization.
-
Secure Store service
The Secure Store Service uses a database to store information. This database must be upgraded as part of a services database attach upgrade. You have to upgrade the data for this service application so that any connections from Excel Services Application and Business Connectivity Services can work with existing passwords.
-
User Profile service
The User Profile service uses databases to store profile, social, and sync information. These databases must be upgraded as part of a services database attach upgrade. You have to attach and upgrade the databases for this service and for the Managed Metadata service before you can upgrade any My Sites.
Note:
My Sites are not available in SharePoint Foundation 2010 or SharePoint Foundation 2013.
Specifically, the following service application databases can be upgraded:
Service application
Default database name
Business Data Connectivity
BDC_Service_DB_ID
Managed Metadata
Managed Metadata Service_ID
PerformancePoint
PerformancePoint Service Application_ID
Search Administration
Search_Service_Application_DB_ID
Secure Store
Secure_Store_Service_DB_ID
User Profile: Profile and Social databases
User Profile Service Application_ProfileDB_ID
User Profile Service Application_SocialDB_ID
User Profile Service Application_SyncDB_ID
The steps to upgrade these service application databases are included in Attach databases and upgrade to SharePoint 2013.
-
-
Considerations for specific services
The following services in SharePoint Server 2013 also require additional steps to enable and configure when you upgrade:
-
Excel Services
You can enable this service by using the Farm Configuration Wizard, but you must make sure that you re-create all trusted data connections. For more information, see Manage Excel Services Trusted Data Providers.
-
InfoPath Forms Service
This service is not part of the Farm Configuration Wizard. If you want to use this service, you can use the Configure InfoPath Forms Services link on the General Application Settings page in SharePoint Central Administration to configure it. If you want to continue using form templates from your previous environment, you can export any administrator-deployed form templates (.xsn files) and data connection files (.udcx files) from your SharePoint Server 2010 environment, and then import them to your new SharePoint Server 2013 environment by using the Export-SPInfoPathAdministrationFiles Windows PowerShell cmdlet. If the URL of the new server differs from the URL of the previous server, you can run the Update-SPInfoPathAdminFileUrl Windows PowerShell cmdlet to update links that are used in the upgraded form templates. For more information, see Configure InfoPath Forms Services (SharePoint Server 2010).
-
Office Web Apps
If you installed Office Web Apps with SharePoint 2010 Products, Office Web Apps will not be available after you upgrade to SharePoint 2013 Products. You must deploy Office Web Apps Server and then connect SharePoint 2013 Products it to after the content databases are upgraded. You do not have to wait until the site collections are upgraded because Office Web Apps Server supports both the 2010 and 2013 site collection modes in SharePoint 2013 Products. For more information, see Office Web Apps.
-
-
Upgrade farms that share services (parent and child farms) to SharePoint 2013
Published: July 16, 2012
Summary: Understand how to upgrade environments that include services farms to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Some services in SharePoint 2010 Products can be shared across multiple farms. A services farm hosts services such as Business Data Connectivity service, Search, and User Profiles that other farms consume. When you upgrade to SharePoint 2013, you first upgrade the services farm, and then upgrade the farms that consume those services. This article describes how to upgrade farms that share services.
Before you begin, make sure that you have reviewed the overall upgrade process described in Overview of the upgrade process to SharePoint 2013.
Note:
This article applies to both SharePoint Server 2013 and SharePoint Foundation 2013. However, only the Business Data Connectivity service is available in SharePoint Foundation 2013. The other services are available only in SharePoint Server 2013.
-
Process for upgrading farms that share services
To upgrade farms that share services, you follow these steps:
1. Starting status: services farm and content farm running SharePoint 2010 Products
In your SharePoint 2010 Products environment, you have one or more content farms that use services from a services farm. The services farm provides cross-farm services and Enterprise Search indexes the content on the content farm.
Pre-upgrade state: 2010 content and services farms
2. Create the SharePoint 2013 services farm
Create a new farm to host the service applications, and install and configure SharePoint 2013.
Create 2013 Services farm
3. Upgrade the Search service application and optionally index the content in the SharePoint 2010 Products content farm
Upgrade the Search service application administration database and run a search crawl against the SharePoint 2010 Products content farm to create the index.
Upgrade the Search service application
4. Upgrade the other service applications.
Upgrade the databases for the other service applications.
Upgrade other service applications
5. Switch the services connection to SharePoint 2013 services farm
Change the SharePoint 2010 Products content farm to consume services from the SharePoint 2013 services farm and retire the SharePoint 2010 Products services farm.
Switch connection to 2013 services farm
6. Create SharePoint 2013 content farm
Create a new farm to host content, and install and configure SharePoint 2013.
Create 2013 content farm
7. Upgrade and index the 2013 content farm
Upgrade the data in the SharePoint 2013 content farm. Configure it to consume services from the SharePoint 2013 services farm. Index the SharePoint 2013 content farm.
Upgrade and index the 2013 content farm
8. Retire the SharePoint 2010 Products content farm
Now that the SharePoint 2013 content farm uses services from a SharePoint 2013 services farm, you can retire the SharePoint 2010 Products content farm.
Retire 2010 content farm
If more than one content farm uses services from the SharePoint 2010 Products services farm, repeat steps 5 through 7 for the remaining content farms until all farms are upgraded and are using services from SharePoint 2013. Except for the order of steps in this process, the process to create and upgrade each farm follows the database-attach upgrade steps outlined in Attach databases and upgrade to SharePoint 2013. This process does not explain how to upgrade site collections. For more information about how to upgrade sites, see Upgrade site collections to SharePoint 2013.
-
Best practices for upgrading to SharePoint 2013
Published: July 16, 2012
Summary: Understand how to get the most out of testing upgrade and how to guarantee a smooth upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
To increase your chances of a successful and faster upgrade to SharePoint 2013, follow these best practices to test and complete an upgrade.
-
Best practices for testing upgrade
To understand your environment before you upgrade it, and to plan for the time that an upgrade will require, you should try one or more trial upgrades. The goal of testing upgrade is to find and fix issues and develop confidence in the outcome before the real upgrade. To develop an accurate trial of the upgrade process from SharePoint 2010 Products to SharePoint 2013, follow these best practices:
-
Know what is in your environment. Do a full survey first.
Document the hardware and software in your environment, where server-side customizations are installed and used, and the settings that you need. This helps you plan the trial environment and also helps you recover if upgrade fails. A worksheet is available to record information about your environment. Download the worksheet at SharePoint 2013 Products Preview Upgrade Worksheet.
-
Make your test environment as similar as possible to your real environment.
If possible, use the same kind of hardware and use the same settings, the same URLs, and so on to configure it. Minimize the differences between your test environment and your real environment. As you introduce more differences, you are likely to spend time resolving unrelated issues to make sure that they will not occur during the actual upgrade.
-
Use real data.
Use copies of your actual databases to run the tests. When you use real data, you can identify trouble areas and also determine upgrade performance. You can also measure how long different upgrade sequences and actions take on different kinds of data. If you cannot test all the data, test a representative subset of the data. Make sure that you find issues with the different kinds and sizes of sites, lists, libraries, and customizations that are present in your environment. If you cannot test all data because of storage concerns, try going over the data in several passes, removing the old trial copies before going on to the next batch.
-
Run multiple tests.
A single test can tell you whether you will encounter big problems. Multiple tests will help you find all the issues that you might face and help you estimate a more accurate timeline for the process. By running multiple tests, you can determine the following:
- The upgrade approaches that will work best for your environment
- The downtime mitigation techniques that you should plan to use
-
How the process or performance may change after you address the issues that you uncovered in your first tests
Your final test pass can help you validate whether you have addressed the errors and are ready to upgrade your production environment.
-
Do not ignore errors or warnings.
Even though a warning is not an error, a warning could lead to problems in the upgrade process. Resolve errors, but also investigate warnings to make sure that you know the results that a warning might produce.
-
Test the upgraded environment, not just the upgrade process.
Check your service applications and run a search crawl and review the log files.
For more information about how to test upgrade, see Use a trial upgrade to SharePoint 2013 to find potential issues and the SharePoint 2013 Products Preview – Test Your Upgrade Process model.
-
Best practices for upgrading to SharePoint 2013
To guarantee a smooth upgrade from SharePoint 2010 Products to SharePoint 2013, follow these best practices:
-
Ensure that the environment is fully functioning before you begin to upgrade.
An upgrade does not solve problems that already exist in your environment. Therefore, make sure that the environment is fully functioning before you start to upgrade. For example, if you are not using web applications, unextend them before you upgrade. If you want to delete a web application in Internet Information Services (IIS), unextend the web application before you delete it. Otherwise, SharePoint 2013 will try to upgrade the web application even though it does not exist, and the upgrade will fail. If you find and solve problems beforehand, you are more likely to meet the estimated upgrade schedule.
-
Perform a trial upgrade on a test farm first.
Copy your databases to a test environment and perform a trial upgrade. Examine the results to determine the following:
- Whether the service application data was upgraded as expected
- The appearance of upgraded sites
- The time to allow for post-upgrade troubleshooting
-
The time to allow for the upgrade process
Try a full search indexing crawl. For more information, see Use a trial upgrade to SharePoint 2013 to find potential issues.
-
Plan for capacity.
Ensure that you have enough disk, processor, and memory capacity to handle upgrade requirements. For more information about system requirements, see System requirements (SharePoint 2013 Preview). For more information about how to plan the disk space that is required for upgrade, see Plan for performance during upgrade to SharePoint 2013.
-
Clean up before you upgrade
Issues in your environment can affect the success of upgrade, and unnecessary or very large amounts of data can affect upgrade performance for both databases and site collections. If you don’t need something in your environment, consider removing it before upgrade. If there are issues detected, try to resolve them before you start to upgrade. For more information, see Clean up an environment before an upgrade to SharePoint 2013.
-
Back up your databases.
Perform a full backup of your databases before you upgrade. That way, you can try upgrade again if it fails.
-
Optimize your environment before upgrade.
Be sure to optimize your SharePoint 2010 Products environment to meet any limits or restrictions, either from your business or governance needs or from the SharePoint 2013 boundaries and limits before upgrade. This will help reduce errors during the upgrade process and prevent broken lists or sites after upgrade. For more information about limits in the product, see SharePoint Server 2010 Capacity Management: Software Boundaries and Limits. For more information about large lists and how to address the lower limit on site collections, see Clean up an environment before an upgrade to SharePoint 2013.
-
(Optional) Set the original databases to read-only if you want to keep your original environment available while you upgrade.
If you expect a long outage period while you upgrade, you can set the databases in the original environment to read-only. Users can continue to access the data but cannot change it. For more information, see Attach databases and upgrade to SharePoint 2013.
-
After upgrade, review the Upgrade Status page and upgrade logs to determine whether you must address issues. Then review the upgraded sites.
The Upgrade Status page reports on the upgrade progress, and the upgrade logs list any errors or warnings that occurred during the upgrade process. Verify all the sites and test them before you consider the upgrade finished. For more information, see Verify database upgrades in SharePoint 2013 and Review site collections upgraded to SharePoint 2013.
-
Defer upgrade for site collections until you can get updated customizations to support 2013 mode.
If you wait until the customizations are available, you can complete the initial upgrade of database and services without significantly affecting use of the existing sites in 2010 mode.
-
Review supported editions and products for upgrading to SharePoint 2013
Published: July 16, 2012
Summary: Understand the editions or versions of SharePoint 2010 Products that you can upgrade to specific editions or versions of SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013 Standard | SharePoint Server 2013 Enterprise
When you plan an upgrade process, make sure that you verify that the intended upgrade path is supported. This article describes the editions and products that are supported and unsupported to upgrade to SharePoint 2013.
Important:
Upgrade from a pre-release version of SharePoint 2013 to the release version of SharePoint 2013 is not supported.
Pre-release versions are intended for testing only and should not be used in production environments. Upgrading from one pre-release version to another is also not supported.
-
Supported topologies
For SharePoint 2013, the only upgrade method is the database-attach upgrade method. Because this method upgrades the databases instead of installing in place over an existing environment, you can attach the databases from a stand-alone installation to server farm (Complete) installation if you want to expand your environment.
Figure: Upgrade to either stand-alone or server farm (Complete) topologies
Before you create your new SharePoint 2013 environment and attach and upgrade the databases, determine the type and size of the environment that you need.
-
Physical topology guidance
The SQL Server topology — in addition to network, physical storage, and caching considerations — can significantly affect system performance. To learn more about how to map your solution design to the farm size and hardware that will support your business goals, see Performance and capacity management. For more information about requirements, see System requirements (SharePoint 2013 Preview).
-
-
Supported editions for upgrade
The following table lists the editions available for SharePoint Server 2010 and the supported and unsupported ending editions when you upgrade to SharePoint Server 2013.
Starting edition
Supported ending edition
Unsupported ending edition
SharePoint Server 2010, Standard edition
SharePoint Server 2013, Standard edition
SharePoint Server 2013, Enterprise edition
You can convert to Enterprise edition after upgrade.
SharePoint Server 2010, Enterprise Edition
SharePoint Server 2013, Enterprise edition
SharePoint Server 2013, Standard edition.
SharePoint Server 2010, Trial edition
SharePoint Server 2013, Trial edition
SharePoint Server 2013, full product
You can convert to the full product after upgrade.
-
Supported cross-product upgrades
The following table lists which Microsoft server products can be upgraded to SharePoint Foundation 2013 or SharePoint Server 2013.
Starting product
Supported ending products
Unsupported ending product
SharePoint Foundation 2010
SharePoint Foundation 2013
SharePoint Server 2013
SharePoint Foundation 2013
SharePoint Server 2013
SharePoint Server 2010
SharePoint Server 2013
SharePoint Foundation 2013
SharePoint Server 2013
SharePoint Server 2013
SharePoint Foundation 2013
Search Server 2010
SharePoint Server 2013 or Search Server 2013
SharePoint Foundation 2013
Project Server 2010 with SharePoint Server 2010, Enterprise Edition
Project Server 2013 with SharePoint Server 2013, Enterprise Edition
-
Plan for upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Find resources about how to plan to upgrade from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
In order to have a successful upgrade to SharePoint 2013, you must plan for the upgrade. This section contains articles that help you plan and prepare for upgrading from SharePoint 2010 Products to SharePoint 2013.
To understand how the upgrade process works, see the articles in Get started with upgrades to SharePoint 2013.
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about how to plan for upgrade.
-
TechNet articles about how to plan for upgrade
The following articles about how to plan for upgrade are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Understand how to minimize downtime and plan for special cases during an upgrade to SharePoint 2013.
Create a plan for current customizations during upgrade to SharePoint 2013
Learn how to identify and evaluate the customizations in your environment, and determine whether you will upgrade them, and how.
Plan to upgrade site collections to SharePoint 2013. Plan for upgrade evaluation sites, notifications, and throttling.
Understand upgrade performance and how to plan for the space and time that is required to upgrade to SharePoint 2013.
Create a communication plan for the upgrade to SharePoint 2013
Create a plan to coordinate and communicate with the upgrade team, site owners and users, and stakeholders.
Clean up an environment before an upgrade to SharePoint 2013
Make sure that your environment is in a healthy state, and delete unnecessary items before you upgrade to SharePoint 2013.
-
Additional resources about how to plan for upgrade to SharePoint 2013
The following resources about how to plan for upgrade to SharePoint 2013 are available from other subject matter experts.
Content
Description
Visit the Resource Center to access videos, community sites, documentation, and more.
-
Determine strategy for upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Understand how to minimize downtime and plan for special cases during an upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
When you upgrade your environment to SharePoint 2013, you want to limit how much downtime that users experience. You might also have a special case that you must address during upgrade. This article describes how to minimize downtime and work with these special cases.
In addition to the information in this article, make sure that you read Review supported editions and products for upgrading to SharePoint 2013 to understand exactly which upgrade situations are valid and lead to successful upgrades.
-
How to minimize downtime during upgrade
The following table lists the techniques that you can use during upgrade to reduce the time that users cannot access their content or to potentially increase upgrade performance.
- Read-only databases You can use read-only databases to continue to provide read-only access to content during the upgrade process. For this approach, you set the databases to read-only on the original farm while the upgrade is in progress on another farm. This method reduces perceived downtime for users. Also, if you encounter a problem with upgrade, you can restore the read-only farm to read-write and restore access to users while you rework your plans before you try upgrade again.
-
Parallel database upgrades You can attach and upgrade multiple databases at a time to speed up the upgrade process overall. The maximum number of parallel upgrades depends on your hardware. This results in faster overall upgrade times for your environment. However, you must monitor the progress and your servers to make sure that the performance is acceptable, and for large databases, parallel upgrades can be slower than single upgrades.
For more information about upgrade performance, see Plan for performance during upgrade to SharePoint 2013 and Use a trial upgrade to SharePoint 2013 to find potential issues.
The instructions for using these techniques are included in Attach databases and upgrade to SharePoint 2013.
- Read-only databases You can use read-only databases to continue to provide read-only access to content during the upgrade process. For this approach, you set the databases to read-only on the original farm while the upgrade is in progress on another farm. This method reduces perceived downtime for users. Also, if you encounter a problem with upgrade, you can restore the read-only farm to read-write and restore access to users while you rework your plans before you try upgrade again.
-
Special cases
You might have other requirements or additional goals that you want to achieve when you perform an upgrade. The following table lists special cases and describes how to approach upgrade for each case.
Case
Upgrade approach
Upgrading an environment that uses forms-based authentication?
Additional steps are required to upgrade when you are using forms-based authentication. For more information, see Configure forms-based authentication for a claims-based web application in SharePoint 2013.
Upgrading very large databases?
In general, very large databases — especially databases that have a large number or large size of document versions inside them — take longer to upgrade than smaller databases. However, the complexity of the data determines how long it takes to upgrade, not the size of the database itself. If the upgrade process times out, it is usually because of connection issues. For more information about how long upgrade might take for your environment, see Plan for performance during upgrade to SharePoint 2013.
Upgrading from the server products in the Office 2007 release?
Use a database attach upgrade method to upgrade to SharePoint 2010 Products, and then upgrade to SharePoint 2013.
Upgrading from SharePoint Foundation 2010 to SharePoint Server 2013?
Attach and upgrade the content databases from SharePoint Foundation 2010 to SharePoint Server 2013.
Changing languages?
You have two choices, depending on whether a single site or your whole environment is changing languages:
- To change the multiple user interface (MUI) language for a specific site, upgrade in the same language, and then install the new language pack and change to that language.
Caution:
You must have the appropriate language packs installed to upgrade any sites based on a localized site definition. If you do not have the new language pack, the sites will not be available. Wait for the new language packs to be released before you try to upgrade those sites.
- To change the installation language for your environment, set up your new environment in the new language, and then attach and upgrade your databases in the new language.
- To change the multiple user interface (MUI) language for a specific site, upgrade in the same language, and then install the new language pack and change to that language.
-
Create a plan for current customizations during upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Identify all customizations in your environment and determine what to change or remove as you upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
If you have extensively customized your sites based on SharePoint 2010 Products, you must determine how you want to handle your customizations when you upgrade to SharePoint 2013. Your approach will vary based on the extent of the customizations, the kind of customization, the complexity of your site, and your goals for upgrading. Before you upgrade, you must identify and then evaluate the customizations in your environment and determine whether you will upgrade them, and how.
-
Identify customizations in your environment
As part of an upgrade testing process, you should create an inventory of the server-side customizations in your environment (solutions, features, Web Parts, event handlers, master pages, page layouts, CSS files, and so on). For more information about how to identify customizations, see Use a trial upgrade to SharePoint 2013 to find potential issues.
You can use the Upgrade Planning worksheet to list specific customizations and then record the results of your evaluation in the next section.
-
Evaluate the customizations
After you have identified the customizations, think about the potential upgrade effect of each one. The following table describes types of customizations and the kind of effect they can have during upgrade.
Category of customization
Types of customizations
Potential effect on upgrade
Visually-affecting
Master pages
Themes
Web Pages
Web Parts
Custom JavaScript
Custom CSS files
Should not affect database upgrade.
For site upgrades: likely to work well in 2010 mode, but need changes to work in 2013 mode.
Test carefully in both modes.
Data structure affecting
Content types
List types
Web templates
Site definitions
Can affect database upgrade if content or list type names conflict with new content or list types in the product, or if templates or definitions are missing.
Non-visually affecting
Web services
Windows services
HTTP handler
HTTP module
Might not be compatible with SharePoint 2013. Test carefully to determine effect. Be prepared to remove or replace.
Now that you know what customizations that you have, and what type that they are, you can decide what to do about them. The following questions can help you evaluate the customizations:
-
Is the customization still valuable?
- Does it serve a useful business need?
- Is it widely deployed and used?
- Does it do something that you cannot do with standard features in the product?
- Does it serve a useful business need?
-
Is the customization well-designed?
- Is it built on supported, predefined site definitions?
- Does it follow best practices for customizations?
- Is it a supported kind of customization, or does it introduce risk into your environment?
- Is it built on supported, predefined site definitions?
As you evaluate every customization, you can also think about your overall approach for customizations. You can choose from among these options:
-
- Keep the customizations, don’t upgrade the sites You can continue to run the site in 2010 mode in the upgraded environment. Although you can use this approach to keep the same functionality, you will be unable to take advantage of the features and capabilities that are available in the new version. Use this approach only temporarily – eventually you must address the issue (such as before an upgrade to the next version of the product).
- Replace or redo the customizations If you want to use new functionality, plan to redesign your sites, or are significantly changing the information architecture, the upgrade is your opportunity to start over with new features, a new look, or a new organization. When you replace or redo customizations, you can take advantage of the new capabilities, change your design slightly if you want, or move to a more manageable design.
- Discard the customizations Replace the customizations by using default functionality. You can reset pages to the default site definitions and remove any Web Parts or features that you no longer want to support. In fact, the site collection health-checker checks for unghosted pages and can reset the pages to the default versions. If you decide to discard any customizations, you must fix any issues that result from removing the customizations in the sites that used them. You can use your customizations inventory to determine which sites require this kind of attention before or after upgrade.
-
Considerations for specific customizations
In addition to your overall decision about how to treat customizations in your environment during upgrade, you must examine specific types of customizations to determine whether you must perform any additional actions to make them work in the upgraded environment.
The following table lists some common customizations and a recommendation for addressing that kind of customization.
Customization type
Recommendation
Site definition
Migrate sites to a supported, predefined site definition, then apply custom features by using solution deployment.
You can also continue to use a custom site definition. You do not have to create a new site definition that is based on SharePoint 2013.
However, if you must perform custom upgrade actions for the definition, you might have to create an upgrade definition file for that site definition. For more information, see Upgrade Definition Files (http://go.microsoft.com/fwlink/p/?LinkId=182339) on MSDN.
“Fabulous 40” application templates
Microsoft is not creating new versions of these templates. Environments that contain sites based on these templates can be upgraded as long as the templates are installed. But there might be issues when you try to upgrade the site collections. Make sure that you test each site before you upgrade the production environment. For more information, see Troubleshoot database upgrade issues in SharePoint 2013.
Feature
Evaluate, then redesign or redeploy if it is necessary.
Workflows and server controls
Depends on the solution. Contact the vendor to discover whether there is an updated solution. If a workflow is compatible with the new version, redeploy.
Event handler
Most event handlers will continue to work without changes. However, if the code for the event handler makes calls to APIs which were deprecated, you will have to rewrite it, and then redeploy it as a feature.
Managed paths (inclusions/exclusions)
Re-create inclusions to make sure that you can access all site collections under those paths.
Exclusions were not used in SharePoint 2010 Products. If you had any remaining from an earlier version, they do not have to be re-created.
Themes
Re-create your themes following the SharePoint 2013 theming guidance, or select a new theme available in SharePoint 2013.
Master pages and CSS files
Rework to accommodate the new user experience.
JavaScript
Test to determine whether any actions are required. In some cases, you might have to adjust the scripts to work with the new page model. Verify that it works in both 2010 and 2013 modes.
Search provider or security trimmer
Test to determine whether any actions are required.
Web Parts
Test to determine whether any actions are required. You might have to adjust the Web Parts to work with strict XHMTL mode.
Test to verify that there have not been changes to any object models or Web services that you call from the Web Part.
If a Web Part is located on a page but not in a Web Part Zone (so that it is, basically, HTML code embedded directly in a page), it will not work if you reset the page to the default template. There is a site collection health rule that will identify files in this status inside a site collection. There is a link from that rule to the page where they can reset to template.
Services
Test to determine whether any actions are required. Redesign or adjust code, as needed.
Authentication providers
Test to determine whether any actions are required. Redeploy the provider with the same provider name (exactly. This includes the letter case) on a test farm and make sure that it works correctly.
Custom search solutions that use SQL syntax
Rework to use FQL syntax and KQL syntax.
Custom search solutions in SharePoint Server 2013 do not support SQL syntax. Search in SharePoint Server 2013 supports FQL syntax and KQL syntax for custom search solutions. You cannot use SQL syntax in custom search solutions using any technologies. This includes the query server object model, the client object model, and the Search REST service. Custom search solutions that use SQL syntax with the index server object model and the Query web service that were created in SharePoint Server 2010 will not work when you upgrade them to SharePoint Server 2013. Queries submitted via these applications will return an error. For more information about how to use FQL syntax and KQL syntax, see Keyword Query Language (KQL) syntax reference and FAST Query Language (FQL) syntax reference.
While you are reviewing customizations in your environment, you should also make sure that the environment is not using any features or elements that are deprecated. For example, Web Analytics from SharePoint 2010 Products are not available in SharePoint 2013 and you should turn them off before upgrading. Also, SQL Server Search queries are not available in SharePoint 2013. For more information, see Changes from SharePoint 2010 to SharePoint 2013.
Some methods of deploying customizations might require additional steps in SharePoint 2013. The following table lists methods of deploying customizations and any issues that you might encounter.
Deployment method
Recommendation
Customizations deployed as MSI files
Contact the vendor for updated files. Most likely, you will have to get a replacement file compatible with SharePoint 2013.
Manually deployed features, files, or changes
You can re-deploy them to the equivalent directory in SharePoint 2013. However, consider packaging them into a deployable solution package for easier administration.
Sandbox solutions
No special steps. Sandbox solutions are upgraded with the content databases.
Solution packages
Redeploy to SharePoint 2013. Make sure that you deploy it to the appropriate directory (/14 or /15), depending on the version.
Note that you can no longer add partial trust solution packages to the \bin directory. Any files deployed to the \bin directory must be full trust. Be sure to test any such solutions to make sure that deploying them in full trust does not introduce security vulnerabilities. Also, update any deployment scripts to make sure that they specify the correct trust level.
For more information, see Install-SPSolution.
Administrator-deployed form templates
You must extract them from SharePoint Server 2010 and redeploy them to SharePoint Server 2013. For more information, see Services upgrade overview for SharePoint Server 2013.
The following kinds of customizations are not supported. If you have any of these customizations in your environment, you must replace them by using a supported kind of customization before you can upgrade. Otherwise, you might experience upgrade issues that cannot be fixed:
- Predefined files, features, or site definitions that were changed.
Warning:
Some predefined file types — such as document icons or actions — can be carried forward in a supportable way, although this does not occur automatically. Do not copy over the old version files as that can cause other issues, instead, make the same changes to the new version file Modifications to other predefined files, such as server-side ASPX pages, will be lost during upgrade if you reset to the site template or if you don’t make the same changes in the new version files. Depending on the files that were changed and the extend of these changes, the upgrade experience can vary significantly.
- SharePoint databases that were changed, either by directly changing data or changing the schema. This includes adding or removing triggers, tables, views, or indexes.
If you have any of these kinds of customizations, remove them and replace them with supported customizations before you attempt to upgrade. This is a best practice for helping to make sure that not only your current upgrade will work, but any future upgrades will go more smoothly. Changing predefined files and databases will remain unsupported.
- Predefined files, features, or site definitions that were changed.
-
Ensure that future customizations follow best practices
Ensure that your environment performs well and follows best practices. Deploy only those customizations that follow the best practices as described on the following page on MSDN: Developer Best Practices Resource Center.
Best practices for upgrading to SharePoint 2013
-
Plan for site collection upgrades in SharePoint 2013
Published: July 16, 2012
Summary: Explains how plan to upgrade site collections to SharePoint 2013 and how to plan for upgrade evaluation sites, notifications, and throttling in SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
When you upgrade from SharePoint 2010 Products to SharePoint 2013, site collections are not upgraded when you upgrade the content databases to a new version. The upgrade process is split to allow site collection administrators to decide when to upgrade their site collections. For a visual overview of the upgrade process, see Overview of the upgrade process to SharePoint 2013.
Server farm administrators can control settings for upgrading site collections, such as settings for upgrade evaluation site collections, notifications, and upgrade throttling. This article helps you plan the settings to use to controlling the upgrade of a site collection.
-
Determine the site collections that farm administrators should upgrade
By default, site collection administrators are in charge of when they upgrade their site collections, and they perform the upgrade themselves. However, under certain circumstances a farm administrator should probably perform the upgrade. For example, for sites that meet the following characteristics, the upgrade team at the farm level should perform tests before upgrade, and potentially upgrade the site collection:
-
Extremely important sites
If a site is very important to your business, farm administrators should carefully test it before they upgrade it, and then upgrade it themselves to make sure that the site collection is available for users as quickly as possible.
-
Very large sites
By default, if a site collection administrator starts to upgrade a site that is larger than 10 MB or with more than 10 subsites, the site is added to the upgrade queue, instead of being upgraded immediately. For very large site collections (larger than 10 GB), we recommend that you have a farm administrator upgrade the site collections instead of allowing the site collection administrators to start the upgrade. This way, the farm administrators can test these sites and then monitor the progress of the upgrade.
-
Highly-customized sites
Carefully test sites that are based on custom site definitions or that have many other customizations before you upgrade them. If there are issues with server-side customizations, then farm administrators should address them, test again, and then perform the upgrade so that they can troubleshoot any issues that occur. If there are issues with the design of a site, a designer and site collection administrator can address them.
Farm administrators can upgrade sites by using Windows PowerShell. For more information, see Upgrade a site collection to SharePoint 2013.
-
-
Plan settings for upgrade notifications, self-service upgrade, and site collection creation
When a site collection is available to upgrade, a status bar on a site indicates that site collection administrators can upgrade it. They can choose to upgrade the site collection then, or be reminded later.
Farm administrators can determine whether to allow site collection administrators to upgrade their sites at all. You can set a property to prevent the site collection administrators from starting to upgrade, which also turns off the notification in the status bar. Then you can perform the upgrades yourself by using Windows PowerShell. If you choose to upgrade some sites centrally, you should have a plan to decide when each site will be upgraded and who will verify the site after upgrade.
Although administrators can upgrade all site collections immediately, we do not recommend this, for the following reasons:
- You would risk that some sites would have unforeseen issues that you’d have to address. This could create or prolong an outage.
- A high volume of issues could arrive at your helpdesk or troubleshooting process when users start to work with upgraded sites at the same time.
You can control settings for site collection upgrade and site creation. You can determine the following:
- Whether the site collection administrator can upgrade the site collection.
-
Which mode (2010 or 2013, or both) can be used when a user creates a site collection.
For example, you might want users to keep creating 2010 mode sites for a while, until most of site collections are upgraded, or you might want to force new sites to be created in 2013 mode so that you don’t have to upgrade them later.
-
Properties that control site collection upgrade and site creation
-
Property
Description
SPSite.AllowSelfServiceUpgrade
Determines whether an upgrade notification can be set for a site collection.
Default is true – notifications are set automatically.
If set to false, the upgrade notification will not appear on the status bar.
SPWebApplication.CompatibilityRange
Determines in which modes a site collection can be created. For example, 2010 mode (14) or 2013 mode (15). The following ranges are available:
- OldVersions Use this range to enable users to create only 2010 mode sites.
- NewVersion Use this range to enable users to create only 2013 mode sites.
- AllVersions Use this range to enable users to create either 2010 or 2013 mode sites.
You can use these ranges or set your range by using the New-Object command to set the Microsoft.Shareoint.SPCompatibilityRange property.
For more information about how to set these properties, see Manage site collection upgrades to SharePoint 2013.
You can also control settings upgrade notifications. You can determine the following:
- Whether to add a link to more information from the Upgrading now status bar.
-
How many days to wait before reminding a site collection administrator about upgrade if they choose Remind me later on the status bar.
If a user clicks Remind me later, the current date is added to the number that is set for the UpgradeReminderDelay and the notification is hidden until that new date occurs. For example, if the setting is 30, then the notification will appear 30 days from the current date.
The following properties control site collection upgrade notifications:
-
Properties that control upgrade notifications
Property
Description
SPWebApplication.UpgradeMaintenanceLink
Adds another link to the upgrading now status message so that users can follow it, and find more information.
Default is empty.
SPWebApplication.UpgradeReminderDelay
Sets the number of days to suspend the upgrade notification in the status bar after a user clicks Remind me later.
Default is 30 days.
If set to 0, then the upgrade notification is not removed from the status bar and the notification cannot be set to Remind me later.
For more information about how to set these properties, see Manage site collection upgrades to SharePoint 2013.
- You would risk that some sites would have unforeseen issues that you’d have to address. This could create or prolong an outage.
-
Plan for upgrade evaluation sites
Site collection administrators can request a preview of their site collection. This preview site is called an upgrade evaluation site collection. An upgrade evaluation site collection enables site collection administrators to see their site‘s content in a new, separate copy of the site that is running on the SharePoint 2013. Unlike visual upgrade in SharePoint Server 2010, the upgrade evaluation site collection is a complete copy of the site collection. It is separate from the original and has its own URL. Actions that the site collection administrators perform in the upgrade evaluation site collection do not affect the original site. Both the original site and the upgrade evaluation site are available for search, and timer jobs that run for all site collections also run on the upgrade evaluation sites.
When a site collection administrator requests an evaluation site collection, the request is added to a timer job (known as “Create Upgrade Evaluation Site Collections”) which runs one time per day. This timer job creates a full copy of the site collection at a unique URL. Upgrade evaluation site collections are set to expire automatically and be deleted. The default time for expiration is 30 days, which can be configured by setting a value for the web application or by changing a value on the evaluation site collection itself.
Farm administrators can choose to prevent users from creating upgrade evaluation sites by setting the SPSite.AllowSelfServiceUpgradeEvaluation property for a site collection.
Timer jobs create and delete upgrade evaluation sites. The following timer jobs are used:
-
Timer jobs for upgrade evaluation site collections
Job name
Description
When run
Create Upgrade Evaluation Site Collections (job-create-upgrade-eval-sites)
Creates upgrade evaluation sites.
Runs daily, between 1:00 and 1:30 AM
Delete Upgrade Evaluation Site (job-delete-upgrade-eval-sites)
Deletes expired upgrade evaluation sites and sends notifications for sites near their expiration date.
Runs daily, between 1:00 and 1:30 AM
Upgrade site collections (job-upgrade-sites)
Upgrades site collections in the queue for a content database.
Runs every 1 minute
You can decide when and how often these timer jobs run, and you can also run them manually.
-
How the upgrade evaluation site collections are created
The Create Upgrade Evaluation Site Collections job timer collects the list of site collections that were queued for evaluation sites, and then copies the sites to new URLs and Site IDs. It also adds the sites to the upgrade queue so that they will be picked up by the Upgrade Site Collections timer job later. To create the copy of the site:
-
- If you have an Enterprise version of SQL Server, the Create Upgrade Evaluation Site Collections job timer takes a snapshot of the database and reads the data from the snapshot to a destination database (with the source database being the default target). This doesn‘t affect the read-only status of the source site throughout the whole process.
- For other versions of SQL Server that do not have snapshot capabilities, the Create Upgrade Evaluation Site Collections job timer backs up a site collection and restores it to a new URL. This makes the source site read-only for the whole duration of the process.
The Upgrade Site Collections job collects the list of site collections that were queued for upgrade and then upgrades the queued sites from oldest to newest. The recently added evaluation site is then upgraded (or at least upgrade is tried).
-
Plan site collection upgrade throttling and queues
To make sure that site collection upgrades do not cause an outage on your farm, there are throttles built in at the web application, database, and content level. This means that even if 100 site collection administrators decide to upgrade their site collections at the same time, only some are run at the same time, and the rest are put into a queue to run later.
Site collection upgrades are throttled:
-
Throttle levels for site collection upgrade
Level
Maximum number of site collections that can be upgraded at a time
Property that controls the throttle setting
Web application
Default is 5 per web application instance.
Additional requests are queued.
SPWebApplication.SiteUpgradeThrottleSettings AppPoolConcurrentUpgradeSessionLimit
Content database
Default is 10 per content database.
Additional requests are queued.
SPContentDatabase.ConcurrentSiteUpgradeSessionLimit
Content of a site collection (size and number of subwebs)
Default is that a site that is more than 10 MB, or has more than 10 subwebs, cannot be upgraded in a self-service manner by the site collection administrator, but must be upgraded by the farm administrator.
SPWebApplication.SiteUpgradeThrottleSettings UsageStorageLimit and SubwebCountLimit
The following illustration shows the relationship between the web application and content database upgrade throttle limits.
Upgrade throttles and the site upgrade queue for web applications and content databases
In this illustration, the content database contains fifteen sites, and all sites were requested to start upgrade.
-
- Because of the web application throttle limit, only five sites can start to upgrade for web application 1 – instance 1 on Web server 1.
- An additional five sites start to upgrade on web application 1 – instance 2 on Web server 2.
- Because of the content database throttle, five sites are sent to the upgrade queue to wait their turn.
You can use the default throttling settings, or you can specify your own values for how many site collections can be upgraded at the same time. Farm administrators can also override throttle settings when they upgrade a site by using Windows PowerShell. Exercise caution when you change these values and make sure that you verify the settings that you want to use in a test environment before you implement them in production. If you increase throttling too much, you could create performance problems in your environment. For example, too many parallel upgrades could affect site rendering. For information about how to change these settings, see Manage site collection upgrades to SharePoint 2013.
-
About site collection modes
In order to make it possible to upgrade site collections separately from upgrading content databases, SharePoint 2013 introduces the concept of site collection “modes” (also known as compatibility levels). Site collections are in 2010 mode in the new environment until they are specifically upgraded to 2013 mode. You can create new site collections in either mode. Although farm administrators can configure this setting, the default setting is to create sites in 2010 mode). When a site collection is in 2010 mode, the user interface resembles the SharePoint 2010 Products interface, and only features that were available in SharePoint 2010 Products are enabled. In 2013 mode, the interface and features are updated to SharePoint 2013.
You have to make sure that the solution packages, features, and other custom components are available for both site modes. For more information, see Create a plan for current customizations during upgrade to SharePoint 2013.
-
Train site collection administrators
It is important to train users about how to upgrade their site collections and how to review their sites in an upgrade evaluation site collection. Educated users are prepared and know what to expect, which will minimize helpdesk support and frustrations.
Inform users about changes and new features. Also, let them know about possible issues that they can expect. For instance, they might have issues with customizations, such as pages that do not display correctly. For information about general upgrade issues, see Review site collections upgraded to SharePoint 2013 and Troubleshoot site collection upgrade issues in SharePoint 2013.
Explain to site collection administrators that their upgrade evaluation sites are copies, and any changes they make there will not persist in their upgraded sites. There is also a notification bar in the preview site that indicates that it is a copy.
By default, site collection administrators control upgrade for their sites. They can use upgrade evaluation site collections to preview the new user interface and features. This gives them time to make sure that everything works correctly, and they can address any issues in their original site before upgrading it. When site collection administrators are ready, they can upgrade their sites.
We recommend that you have a plan and set a time limit for how long to allow site collection administrators to postpone upgrade of their sites. For example, each site collection administrator may be given 90 days to work with his or her site collection administrators to evaluate and then upgrade their sites. This time limit makes sure that users are given a reasonable time to become familiar with the new user interface and to resolve any issues in their sites. Ensure that you communicate the time limit to the users, and that they know that you can force through an upgrade of all sites. Also, you can use a Windows PowerShell command to check the compatibility level for sites in a content database so that you can see how many sites are in 2010 mode and how many are in 2013 mode. For more information, see Manage site collection upgrades to SharePoint 2013.
It is important to tell site collection administrators that as long as sites use the 2010 mode, new features will not be available. However, as soon as sites are upgraded to the new version, application features automatically appear.
-
Plan for performance during upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Understand upgrade performance and how to plan for the space and time that is required to upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
An important part of planning an upgrade from SharePoint 2010 Products to SharePoint 2013 is determining how long the upgrade process will take and how much storage space will be required. Every environment is unique and includes different hardware capabilities and different database and site characteristics. The space and the length of time required to run an upgrade will vary greatly depending on your environment. The best way to estimate these factors is to perform one or more trial upgrades, and then review the space and time that it took. For more information about how to perform a trial upgrade, see Use a trial upgrade to SharePoint 2013 to find potential issues.
-
About upgrade performance for SharePoint 2013
One of the main reasons that database upgrade and site collection upgrade are now separate actions for SharePoint 2013 is to give you more control over upgrade performance.
-
How upgrade works for SharePoint 2010 Products
During an upgrade to SharePoint 2010 Products, when the database was upgraded, all of the site collections in that database were also upgraded. That meant that for certain steps, such as activating features or updating the Quick Launch control, the upgrade process needed to perform the step repeatedly for each site collection in a database before the database upgrade was completed.
Upgrade process for SharePoint 2010 Products
-
How upgrade works for SharePoint 2013
When you upgrade to SharePoint 2013, the database upgrade no longer starts these site-specific steps. Therefore, the database upgrade stage is much faster. Some tests have found that a database can be upgraded to SharePoint 2013 in two-thirds of the time that it took to upgrade the same data to SharePoint 2010 Products. These site-specific steps must still be performed. However, they are performed only when the site collection is upgraded, and then only for that site. Because each site collection is upgraded independently, you can manage the performance effect of those upgrades on your environment.
Upgrade process for SharePoint 2013 Products
For SharePoint 2013, you can control the performance effect of site collection upgrades by controlling how many sites can be upgraded at a time. There is a site collection upgrade queue that maintains a list of site collections currently requested to be upgraded. And there are throttles on the content database and web application levels to control how many site collection upgrades can occur at a time. For more information about these controls, see Plan for site collection upgrades in SharePoint 2013 and Manage site collection upgrades to SharePoint 2013.
In addition to planning for performance during upgrade, you must also plan for the performance of your production environment after upgrade. Test your planned environment to make sure that you can support your environment by using the hardware that you have planned.
-
-
Estimate the space that you must have for the upgrade
As the databases are upgraded, they expand temporarily. Also, many transactions occur while the upgrade process runs. Therefore, you must make sure that the log files have room to expand to accommodate the changes that are occurring. Therefore, you have to plan for growth in both the databases and the log files.
-
Database growth
Because of the changes in table structures in the new version, the databases grow temporarily while the data is reorganized. This space can be recovered after upgrade, but you should make sure that there is room for the databases to grow up to 50 percent larger than their current sizes during upgrade (be aware that after upgrade, you can reduce the database again to recover much of this space).
You should also make sure that there is room on your database servers for your databases to grow over time with typical use. To check how large your databases currently are, use Enterprise Manager in SQL Server.
-
Transaction log growth
In addition to database space, you must also have room for the transaction log files for the databases. These log files must grow quickly to accommodate the number of changes occurring in the databases
In very large environments, there is a possibility that the default growth rate for the transaction log files (10 percent) is not enough to keep up with the upgrade process. This can cause a time-out. Again, a trial upgrade is the best way to determine whether the transaction log files can keep up with the upgrade process. If your environment is very large, or if the process timed out during a trial upgrade, consider expanding the SQL Server transaction log files beforehand to make sure that you have room for the number of transactions that must be processed. For more information about how to expand the SQL Server transaction logs, see Expanding a Database (SQL Server 2008 R2).
-
-
Estimate how long the upgrade will take
With your disk space estimates in hand, and some testing done, you can now calculate a rough estimate of how long the actual upgrade process will take. Upgrade times vary widely among environments. The performance for an upgrade depends greatly on the hardware being used, the complexity of the sites, and the particular characteristics of your implementation. For example, if you have many large document libraries, these may take longer to upgrade than a simpler site.
Factors that influence performance for upgrade are described in the following list.
-
Environment factors The following factors can affect the performance for both database and site collection upgrade:
- Simultaneous upgrades
- SQL Server disk input/output per second
- SQL Server database to disk layout
- SQL Server temporary database optimizations
- SQL Server CPU and memory characteristics
- Web server CPU and memory characteristics
- Network bandwidth and latency
- Simultaneous upgrades
-
Database factors The following factors can affect the performance of database upgrade. The number of:
- Site collections
- Subwebs
- Lists
- Rowspan within lists
- Document versions (number and size)
- Documents
-
Links
Plus the overall database size itself.
- Site collections
-
Site collection factors The following factors can affect the performance of site collection upgrade. The number of:
- Subwebs
- Lists
- Activated upgrading features
- Document versions (number and size)
- Documents
- Links
- Subwebs
How your data is structured can affect how long it takes to upgrade it. For example, 10,000 lists with 10 items each will have a longer upgrade time than 10 lists with 10,000 items. The actions required to upgrade the list infrastructure must be performed for each list, regardless of the number of items. Therefore, more lists equals more actions. The same goes for most of the items under database factors or site collection factors.
The structure of your hardware can also have a big effect on performance. Generally, the database server performance is more important than web server performance, but underpowered hardware or connectivity issues at either tier can significantly affect upgrade performance. Web servers have a significant part to play in database upgrade performance, mainly by issuing the commands to make data and structural changes in the databases. The database servers have to process those changes and work with a large set of data for every command. Web server performance becomes a bigger issue during site collection upgrade, when the upgrade process iterates several actions for each site collection (which might occur for multiple site collections at a time). Site collection upgrade also affects the database server as each action must occur in SQL Server.
The best way to estimate overall time is to do a trial upgrade of the data, and then review the upgrade log files. The log files contain the duration for an upgrade — look for Total Elapsed Time at the bottom of the upgrade log file. Use this time to estimate the duration for your full set of content. You can also use the log files to check your progress during the upgrade process. The upgrade.log file is located at %COMMONPROGRAMFILES%\Microsoft Shared\web server extensions\15\LOGS.
To determine durations from the logs, examine:
- The times spent in each upgrade sequence per log.
- The times spent per upgrade action instance per log.
- The minimum, maximum, and average times.
Make sure that you separate out the database upgrade times from time that is spent upgrading site collections for accurate planning. Perform multiple trial upgrades to guarantee more accurate data. Then collate the data from multiple tests to determine likely performance per sequence, action, and database. You can use the Get-SPUpgradeActions cmdlet in Windows PowerShell to see how many actions occur for the farm. For more information, see Get-SPUpgradeActions.
The estimate you arrive at based on your trial upgrade is for the actual upgrade process for the data. It does not include all of the steps that you have to perform before and after this step, which can take more time than the upgrade of the data itself. When estimating how long the upgrade will take, in addition to the time that is required for data to be processed, you must also estimate how long the activities during the upgrade phases will take.
Consider the following factors:
- Creating custom elements Upgrading Web Parts or re-doing custom templates to take advantage of new features will take some time. The process of creating custom elements should begin early, during the evaluation phase of your project.
- Backing up the databases You must perform a full backup — not a differential backup — of your databases for a database-attach upgrade. For large environments, this step can take significant time. In particular, if you are backing up to a network location, network latency issues can slow this process down.
- Creating service applications and configuring services Creating service applications and configuring services does not take long. However, if you must contact a database administrator to create the databases for you before you create service applications, you might need a day or two of lead time.
- Running a search crawl on all content For large sites, this step can take more than 24 hours. Unless you run a crawl, searches cannot return results because the index is not upgraded.
- Verifying the environment after upgrade For large environments, it can take a while to verify the environment and check the site collections to make sure that they are in a good state before you allow users access to them. For more information, see Verify database upgrades in SharePoint 2013.
For site collection upgrade, consider the following factors:
- Verifying sites and making changes Allow enough time for users to validate their sites after the upgrade. This may take several days. For more information, see Review site collections upgraded to SharePoint 2013.
Additional factors in your environment can also contribute to longer upgrade times. They include the following:
- Very large document libraries A document library with more than 250,000 documents all in the root of the document library (instead of in folders) will take a long time to upgrade, and the upgrade might not be successful. Following the guidelines for using folders to break up large document libraries can help you manage the library size. For example, if you rearrange the same document library so that the 250,000 documents are divided into 125 folders, it should upgrade more easily.
-
Very large databases Databases larger than 100 GB can take a long time to upgrade.
If you have content databases that are larger than 100 GB and include mixed site types (such as My Sites and team sites together with published sites), we recommend that you divide them up into smaller databases that contain a consistent type of data before you run the upgrade.
Note:
My Sites are available only with SharePoint Server, not SharePoint Foundation.
You can use the Move-SPSiteWindows PowerShell cmdlet to move sites between databases. For more information, see Move-SPSite.
Be sure that you are following the capacity planning guidelines from the previous and new versions before you attempt the upgrade. If you have exceeded the guidelines for best performance, the upgrade process might take longer, or it might fail (for example, the process might time out repeatedly on the same large document library). If your deployment does not meet the recommended capacity guidelines, consider whether you have to do some work to meet those guidelines before you try the upgrade. Again, a trial upgrade can help you with that decision.
-
Wide lists (lists with many columns)
Wide lists are lists with more columns than fit in a single rowspan in the content database. These lists can take longer to process during upgrade, or might not upgrade. For more information, see Clean up an environment before an upgrade to SharePoint 2013.
-
Sites with many subwebs
Site collections that contain hundreds or thousands of subwebs will take much longer to process during site collection upgrade. For example, a site collection with thousands of subwebs might take many hours instead of many minutes, or longer, to upgrade.
-
Communications requirements
You have to notify the users and your team of the upgrade schedule, and give them time to do their tasks. For more information, see Create a communication plan for the upgrade to SharePoint 2013.
-
Managing system center alerts and alarms
You have to monitor system performance during upgrade, but you will not have to monitor specific features. Pause any unnecessary alarms and alerts from Microsoft Systems Center Operations Manager or Microsoft Operations Manager, and then turn them on again after upgrade.
-
Turning SQL Server mirroring and log shipping (or AlwaysOn Availability Groups) on/off
You should turn off mirroring and log shipping before you upgrade, and then turn them on again after you are sure that your environment is running correctly after the upgrade. We recommend that you do not run mirroring or log shipping during upgrade, because this creates additional load on the servers that are running SQL Server and also wastes resources mirroring or shipping temporary data.
This also applies to AlwaysOn Availability Groups in SQL Server 2012.
Test the upgrade process to discover how long it may take, then create a schedule for the upgrade operations and test that to determine your timeline. You should include the time that that is required to do the pre-upgrade and post-upgrade steps in your operations timeline: If it takes 5 hours to back up your databases, you must include that time in your timeline. Also include buffer time in case you have to restore or recover — you should determine both your planned outage (realistic case) and your emergency outage (worst case) timelines.
-
-
Environment performance after upgrade
After you complete an upgrade, the environment will likely experience some lag in performance as it works through the changes. Make sure that you confirm the actual performance against your expected performance after upgrade to make sure that your new farm is performing within acceptable bounds. Check SQL Server responsiveness: is the disk queue length too long? Are CPU and memory usage too high? Also look for web and application server responsiveness: is the number of requests per second (RPS) acceptable? What about page load time (initial and secondary page requests). Review your overall environment performance and make adjustments as needed.
-
Create a communication plan for the upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Communicate timelines, requirements, and how to obtain help with site owners and users during upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
It is important that you communicate with users during the upgrade process from SharePoint 2010 Products to SharePoint 2013. Site users have to know what to expect when they visit their sites again after you have upgraded the environment. Site owners have to know how they can help prepare for upgrade and what they have to do to upgrade their site collections in SharePoint 2013 and My Sites in SharePoint Server 2013. Both site users and site owners have to know when the upgrade will occur. As part of the planning process, determine the following:
- Who are the members of the upgrade team, what other stakeholders are involved, and who will be affected by the upgrade?
- What information must the upgrade team have, and when?
- What information must site users and other stakeholders have, and when?
This article describes how to create a communication plan so that the upgrade team, stakeholders, and users know what to expect before, during, and after the upgrade.
- Who are the members of the upgrade team, what other stakeholders are involved, and who will be affected by the upgrade?
-
Who is a member of the upgrade team?
For small deployments in which sites were not customized extensively, the upgrade team might consist of only one person. For larger deployments, on the other hand, several people with different roles can be required, as described in the following list:
- Server administrators Server administrators perform most of the upgrade tasks. There must be at least one server administrator on the upgrade team because running the Setup wizard requires someone who is a member of the Administrators group on each front-end web server.
Note:
Farm administrators might not be local administrators for the server.
- Database administrators If you have a separate database administration team, you must coordinate with them to schedule the upgrade and perform the upgrade.
- Server security teams You must coordinate with your security teams, such as the Active Directory Domain Services (AD DS) team, to verify accounts and permissions or to take advantage of the new policy settings that you can apply for SharePoint 2013.
- Network teams You must coordinate with the network teams, especially if you must switch DNS to point to your new farm, or add the new servers to the network infrastructure.
- Client deployment team Communicate with client deployment teams to coordinate deployments of new client and server applications. Client deployment might have to occur before you upgrade, or it might be an option for users after their sites are upgraded.
- Services administrators You must communicate with the administrators for service applications, such as the Business Data Connectivity service, to make sure that they are ready for the upgrade and they can review or reconfigure the appropriate settings in the new version.
- IT or application Helpdesk leadership and personnel If you have a helpdesk for your company, make sure that they know about the timing for the upgrade and are prepared for questions after upgrade. Helpdesk should be a key stakeholder for planning and testing so that they can be understand the potential changes from an upgrade and the effect that it will have on users.
- Site collection owners You must notify site collection owners when the upgrade process is about to occur. Warn them about any issues that you find when you run the pre-upgrade checker or when you upgrade their sites. You must also communicate with site collection owners about their role in upgrade. Site collection owners can upgrade their own sites in SharePoint 2013. Site collection owners can run health checks and review upgrade evaluation sites before they upgrade their sites.
- Site designers and developers and third-party solution providers If you have custom templates, Web Parts, Web services, or other custom elements that are associated with your sites, you must work with the associated site designers and developers or third-party solution providers. Because custom elements can fail or perform differently in an upgraded environment, you have to make sure that designers or developers can create new versions of these custom elements or verify that these elements were upgraded correctly. Because their work can have a big influence on the upgrade schedule, work with these stakeholders early in the process. For more information about potential issues with custom elements, see Use a trial upgrade to SharePoint 2013 to find potential issues.
- Site users Although you do not have to include site users in making decisions about the upgrade process, you must tell site users when it will occur and what they should expect.
- Sponsors and other stakeholders Other people in your organization might be involved in the upgrade planning process. Make sure that you include them in your communication plan appropriately.
Note:
An upgrade team can include one or more members in each role, depending on your organization.
- Server administrators Server administrators perform most of the upgrade tasks. There must be at least one server administrator on the upgrade team because running the Setup wizard requires someone who is a member of the Administrators group on each front-end web server.
-
When and what to communicate to the upgrade team
In general, the server administrators and service application administrators set the timeline for upgrade, and site owners are notified only when the process is about to begin. However, because team members have their own tasks to perform at particular points in the overall upgrade process, it is very important that you have a solid plan to communicate the progress of the upgrade to all team members so that everyone knows when it is time to perform their particular tasks.
The whole upgrade team must work together to determine the dates and times to perform the upgrade. We recommend that you choose an upgrade window to occur when site usage is lowest. For small single-server deployments, upgrade may be completed in less than a day. Larger deployments can take more time, up to a weekend. There is no way to determine the precise length of time that will be required to upgrade any particular site collection. Because of this, it is very important to communicate with other team members involved in the upgrade process in addition to users. The day or days that you choose for upgrading should be far enough in the future that the upgrade team has enough time to complete all of the preliminary steps. When you plan the timeline, make sure that you schedule time to validate the upgraded sites and time to implement any changes or do any work to re-brand sites.
It is important to communicate with site owners, designers, and developers at the following points during the upgrade process:
- Before the trial upgrade so that they know the general timeline and their roles in the process.
- After you perform a trial upgrade to find issues. For example, issues such as customized site templates or custom Web Parts should be reported to the appropriate site owner, designer, or developer before you schedule the upgrade, to give them time to investigate the issues and take preliminary steps. Or a developer might decide that it would be prudent to rebuild a Web Part before the upgrade occurs. And site owners might want to note any customizations that were done to their sites, such as site templates and changes to core Active Server Page Extension (ASPX) files.
- After the environment is upgraded so that they can review the sites and make any changes that are needed.
- When you are ready for them to upgrade their site collections.
- Before the trial upgrade so that they know the general timeline and their roles in the process.
-
When and what to communicate to site users
It is equally important to communicate with the users of the sites to tell them about the following issues:
- When the environment will be upgraded In particular, you must also inform them if their sites will be unavailable during the upgrade.
- When their sites will upgraded Site collection owners should communicate to their site users about the timeline for upgrading the site collection. If you, as a server farm administrator, are upgrading a site, you should communicate when that will occur.
- How the upgrade might affect them and what they should know about the new environment For example, the site will look different and function slightly differently in the new user interface. You can also point them to available content, such as What’s New articles or training materials, to learn about the new version. For more information about feature changes, see What’s new in SharePoint Server 15 Beta.
- How to obtain help If they find an issue with their site after upgrade, how can they obtain help in addressing it?
You can use the new system status bar in the site collections to notify users of these items. For more information about how to set notifications for the status bar, see Plan settings for upgrade notifications, self-service upgrade, and site collection creation in the article Plan for site collection upgrades in SharePoint 2013.
- When the environment will be upgraded In particular, you must also inform them if their sites will be unavailable during the upgrade.
-
Clean up an environment before an upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Make sure that your environment is in a healthy state, and delete unnecessary items before you upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Before you start to upgrade from SharePoint 2010 Products to SharePoint 2013, you should make sure that your environment is functioning in a healthy state and that you clean up any content that you do not have to upgrade. You can also take the time to remove or rearrange content so that you will have the structure that you want after you perform the upgrade.
-
Items to clean up
Many of these items can be removed or repaired by using Stsadm command-line tool or Windows PowerShell cmdlets.
Important:
To use the Stsadm command-line tool, you must be a member of the Administrators group on the local computer.
To use Windows PowerShell cmdlets in the SharePoint 2013 Management Shell, you must have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Delete unused or underused site collections and subwebs
You do not want to upgrade content that you do not have to keep. If it was unused for a long time and is not needed in the future, back it up, and then delete it to free storage and administrative resources, improve upgrade performance, and reduce upgrade risk. Be sure to communicate with site owners or organizational contacts regarding the site status — you want to make sure that the site is not needed before you delete it (for example, you do not want to delete sites that are required for compliance, such as emergency procedures, even though they may not be frequently updated).
For more information about how to delete site collections and subwebs, see the following articles:
- Remove-SPSite
-
-
Check large lists (lists with lots of data)
-
By default, large list query throttling is turned on in SharePoint 2010 Products. This behavior has not changed in SharePoint 2013. If a list is very large, and users use a view or perform a query that exceeds the limit or throttling threshold, the view or query will not be permitted. If you are upgrading content from the server products in the Office 2007 release, check any large lists and have the site owner or list owner address the issue. For example, they can create indexed columns with filtered views, organize items into folders, set an item limit on the page for a large view, or use an external list. For more information about large list throttling and how to address issues with large lists, see Manage lists and libraries with many items on Office Online.
-
Delete excess columns from wide lists (lists with too many columns) or remove wide lists
Wide lists are lists with more columns than fit in a single rowspan in the content database. During upgrade, the underlying storage in the database is changed to a sparse table structure, and a very wide list can cause upgrade to fail. Use the Test-SPContentDatabase command in Windows PowerShell to look for wide lists in the content databases and then remove excess columns, or remove the wide list before you upgrade.
For more information about maximum column sizes in a list, see Column limits.
-
Consider moving site collections into separate databases
If you have 5,000 or more site collections in a database, consider breaking them out into multiple databases. In SharePoint 2010 Products, there was a default warning at 9,000 site collections and a hard limit at 15,000 site collections. In SharePoint 2013, these values change to 2,000 site collections for the warning and 5,000 site collections for the limit. To avoid errors during upgrade or broken sites after upgrade, we recommend that you move some site collections into separate databases. If you have multiple content databases, you can also speed up an upgrade process by upgrading multiple databases in parallel.
For more information about site collection limits, see Content database limits. For more information about how to move site collections to a new database, see Move site collections between databases in SharePoint 2013.
-
Remove extraneous document versions
Large numbers of document versions can slow down an upgrade significantly. If you do not have to keep multiple versions, you can have users delete them manually or use the object model to find and remove them. For more information about how to programmatically remove extraneous versions, see Versions Web Service on MSDN.
-
Remove unused templates, features, and Web Parts
First, verify that no sites are using the template, feature, or Web Part. You can use the Stsadm -o EnumAllWebs operation with the –includefeatures and –includewebparts parameters to identify these customizations in your environment. This operation identifies Web Parts, features, event handlers, and setup files that are being used in your environment. The EnumAllWebs command also specifies which files are used by which sites. Changes were made to the EnumAllWebs command in the February 2011 Cumulative update to make it return both site collection and web-level features. For more information, and to get the cumulative update, see Description of the SharePoint Foundation 2010 cumulative update package (SharePoint Foundation server-package): March 3, 2011.
You can remove a feature during site collection upgrade. Simple features can also be removed by deprecating them in the template. You can use feature upgrade to remove more complex features. For more information, see Upgrading Features and Feature Upgrade Overview on MSDN.
For more information about how to identify customizations in your environment, see Use a trial upgrade to SharePoint 2013 to find potential issues. If customizations are not being used, delete them. For more information about how to manage these kinds of customizations, see Features and Templates and Solutions and Web Part Packages on MSDN.
-
Remove PowerPoint Broadcast sites
These sites and site templates are not available in SharePoint 2013 because the Office Web Apps Server are now installed separately from the SharePoint 2013 environment. Sites based on these templates will not work in SharePoint 2013. Remove these types of sites before you upgrade.
You can use the Get-SPSiteWindows PowerShell command together with the following options to find these sites:
Get-SPSite | Where-Object{$_.RootWeb.Template -eq “PowerPointBroadcast#0”}
This will return all sites that use that template.
You can also use the Get-SPSite and Remove-SPSiteWindows PowerShell commands together with the following options to remove these sites:
Get-SPSite | Where-Object{$_.RootWeb.Template -eq “PowerPointBroadcast#0”} | Remove-SPSite
Be sure to back up these sites before you remove them. For more information, see Get-SPSite and Remove-SPSite.
-
Finish Visual Upgrades in SharePoint 2010 Products
During an upgrade from the server products in the Office 2007 release to SharePoint 2010 Products, you could allow site owners to use Visual Upgrade to keep sites in the old experience on the upgraded environment. When you upgrade to SharePoint 2013, all sites that are still in the old experience in SharePoint 2010 Products are automatically upgraded to the 2010 experience. If you want the opportunity to address any issues and review the sites before they are switched to the new experience, upgrade them to the new experience in your SharePoint 2010 Products environment and review them before you upgrade them to SharePoint 2013. We recommend that you finish visual upgrades before you upgrade to SharePoint 2013. Finishing visual upgrades before you upgrade provides the following benefits:
- You can address issues while you still have the server products in the Office 2007 release components available.
- You can have users be involved in reviewing and fixing issues in their sites.
- You can roll back to the old experience temporarily if it is necessary. You cannot roll back when you are in the SharePoint 2013 experience.
- You avoid adding potential errors to the upgrade process. The fewer operations occurring during upgrade, the better. Trying to troubleshoot errors is more difficult when you have more processes involved. And users might think that upgrade has caused an issue when it’s really the experience changing to the new version. If you have an issue with how the site interface is displaying, how will you know whether it is an old issue from the site that was forced through visual upgrade, a problem with the 2010 mode in SharePoint 2013, or a problem with a new CSS file?
To check for sites in the old experience, on the SharePoint 2010 Products environment, you can use the Get-SPSite Windows PowerShell command.
To check for and upgrade sites still in the old experience in the SharePoint 2010 Products environment by using Windows PowerShell
- securityadmin fixed server role on the SQL Server instance.
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2010 Products.
- Click SharePoint 2010 Management Shell.
-
At the Windows PowerShell command prompt, type the following command to return all site collections that are in or have subwebs in the old experience:
Get-SPSite | ForEach-Object{$_.GetVisualReport()}
-
At the Windows PowerShell command prompt, type the following command to upgrade those sites to the new experience:
Get-SPSite | ForEach-Object{$_.VisualUpgradeWebs()}
For more information, see Get-SPSite and Manage visual upgrade (SharePoint Server 2010).
-
Repair data issues
Make sure that you repair all issues in your databases or site content before you upgrade. In particular, check the following items:
-
Check databases for corrupted data
Clean up your databases to remove any orphaned sites or other corrupted data, such as a corrupted list. Consider defragmenting if you have removed sites or subsites from the database. For more information, see:
-
Check databases for duplicate or orphaned site collections
Make sure that site collections exist in only one content database. Occasionally, site collections can leave behind duplicate or orphaned references in old content databases if they are moved to new databases, or if a copy of a database was attached to the farm, or if there was an error when a site collection was provisioned. If a site collection is referenced in more than one content database or there is more than one instance of the site collection in a content database, it can cause issues when you upgrade by using the database attach upgrade method. If you upgrade a duplicate version of the site collection first, the site map in your configuration database might end up pointing to that version of the site instead of the current version.
Before you upgrade, use the Enumallwebs operation in stsadm command-line tool to discover which sites are in which content databases and compare the results. Also, examine each site collection in the results and check whether it is listed as missing in the site map. Being listed as missing indicates that it is an orphaned site. For more information, see Enumallwebs: Stsadm operation. If you find duplicate or orphaned sites, you can use the Remove-SPSite cmdlet in Windows PowerShell to remove the duplicate or orphaned sites from the database.
For more information, see Remove-SPSite.
-
Check variations
In publishing environments, check for any variations that must be fixed. For more information, see Variationsfixuptool: Stsadm operation.
-
How to make structural changes
To make structural changes to your environment, such as moving site collections or changing how your databases are allocated, you can use the following methods:
-
Move-SPSite Use this to move site collections between databases. If a database is very large or contains lots of site collections, you can move sites to address this to make upgrade more efficient. Also, you can move all collaboration sites into one database and all My Sites into another to make the upgrade administration easier for those different sets of sites. You can also use this operation to divide large databases if they contain multiple site collections. This can also help increase upgrade efficiency.
For more information, see Move-SPSite.
- Export-SPWeb and Import-SPWeb Use this method to move subwebs or site collections inside a farm or between farms. For more information, see Export-SPWeb and Import-SPWeb.
-
-
Test and troubleshoot an upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Find resources about how to test and troubleshoot an upgrade from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Before you upgrade from SharePoint 2010 Products to SharePoint 2013, you should take time to test an upgrade process and understand the issues that you might face in an actual upgrade. After you perform a test upgrade, or after you upgrade your actual databases, you might find issues that have to be addressed. After you address issues, you can restart the upgrade to try again.
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about how to test and troubleshoot upgrade.
-
Downloadable resources about how to test and troubleshoot upgrade
Download the following content for information about how to test and troubleshoot upgrade.
Content
Description
SharePoint 2013 Products Preview – Test Your Upgrade Process model
See a visual display of information about how to test the upgrade process.
Use this worksheet to record information about your environment while you test upgrade.
-
TechNet articles about how to test and troubleshoot upgrade
The following articles about how to test and troubleshoot upgrade are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Use a trial upgrade to SharePoint 2013 to find potential issues
Find out how to plan for success by testing upgrade by using your actual data in either a physical or virtual environment.
Follow these recommendations to troubleshoot any issues that occur during database-attach upgrade. You can also look up common issues and discover how to address them.
Troubleshoot site collection upgrade issues in SharePoint 2013
Follow these recommendations to troubleshoot any issues that occur during a site collection upgrade. You can also look up common issues and discover how to address them.
Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013
If you encounter errors during upgrade, you can address them by using the troubleshooting article, and then use this article to restart or resume upgrade.
-
Additional resources about how to test and troubleshoot upgrade
The following resources about how to test and troubleshoot upgrade are available from other subject matter experts.
Content
Description
Upgrade and Migration Resource Center for SharePoint 2013 Products
Visit the Resource Center to find additional information about upgrades to SharePoint 2013.
-
Use a trial upgrade to SharePoint 2013 to find potential issues
Updated: October 16, 2012
Summary: Prepare for upgrade to SharePoint 2013 by testing the upgrade process on copies of real data.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Before you start an upgrade from SharePoint 2010 Products to SharePoint 2013, you should test the upgrade process to make sure that you know exactly what you have to do to have a successful upgrade. A trial upgrade to test the process can reveal the following issues:
- Whether an upgrade plan will work or if you must make adjustments.
- The customizations that are in your environment so that you can plan to deal with them during upgrade.
- Whether you should upgrade your hardware to make an upgrade run more efficiently and more quickly.
- The timing for upgrade, or how long upgrade will take for your environment.
- What you must plan for, operationally — for example, resources to have available.
In addition, you can use the trial upgrade to become familiar with the upgrade tools and the process itself so that you know what to expect when you perform the actual process. Through testing, you can discover the following issues:
- What does the upgrade user interface look like? How do you know when you have finished one phase and are moving through another?
- Where are the log files, and how do you read them? What information do they provide?
- Whether you must adjust any scripts or commands that are used during the upgrade process, especially if you are relying on any scripts that were used in upgrading to SharePoint 2010 Products.
- Whether you have the right plan to address any outages.
This article describes basic steps for testing upgrade, and it gives recommendations for reviewing the results and adjusting an upgrade plan based on what you learn during the tests.
In addition, the following resources can be helpful when you test the upgrade process:
-
SharePoint 2013 Upgrade Worksheet
Download the SharePoint 2013 Products Preview Upgrade Worksheet and use it to record information about your environment while you test upgrade.
-
SharePoint 2013 – Test Your Upgrade Process model
Download the SharePoint 2013 Products Preview – Test Your Upgrade Process model poster to see a visual display of information about how to test the upgrade process.
- See best practices for testing the upgrade process in Best practices for upgrading to SharePoint 2013.
- Whether an upgrade plan will work or if you must make adjustments.
-
Set up a test environment
You can use either virtual or physical hardware to test the upgrade process. Every environment is unique. Therefore, there are no general guidelines for how long upgrade will take or how difficult a particular customization will be to upgrade. The best way to gauge how upgrade will go is to perform a series of trial upgrades.
Here are some things to consider when you create your test environment:
- Make your test farm as similar as possible to your real farm — for example, hardware, software, and available space.
- Use the same URLs in your test farm as in your real farm. Otherwise, you will waste time diagnosing issues that relate to the URLs that will not occur in the real upgrade. You can do this by using the same URLs and testing only from computers that have host file changes.
-
Use the different computer names for your web and application servers.
This will prevent Active Directory Domain Services (AD DS) conflicts.
-
Use separate servers that run SQL Server for your test farm
If you are using the same servers that run SQL Server for your test and production farm, you can affect the performance of your production farm while you run your tests. We recommend that you use different SQL Server computers (not just instances) for your production and test farms.
-
Use the same database names in your test environment.
That way, you can validate any scripts that you use to manage your environment. Again, make sure that you use separate servers that are running SQL Server or you risk affecting your production environment.
- Be sure that you transfer all the settings and customizations to the test environment. The section Identify and install customizations provides information about collecting this information.
Make sure that actions that you take in the test environment do not affect the live environment. Be cautious with the following:
-
External data connections
Even though you are working with a copy of the environment, the link to the data source is real. Changes that you make to the data in the test environment affect the production environment.
-
Running commands against a live database still in production
Make sure that you use copies of your databases for testing, not a live version in your production environment. For example, if you run Test-SPContentDatabase against a live database, instead of a copy, you might affect performance on your production environment.
-
Using a virtual test environment
-
When you test by using a virtualized environment, you do not have to have lots of hardware. You can replicate your environment by using just two servers that are running Hyper-V. One server has images for the front-end web servers and application servers, and the other server has images for the database servers.
However, virtual environments might not have the same performance metrics as physical environments. If your production environment is physical, you must consider this difference when you calculate the time that is required to upgrade your production environment. Generally, you can get better performance estimates if you use a physical server for SQL Server. Make sure that it has similar performance specifications to your server that runs SQL Server in your production environment.
Distribution of servers in a virtual test environment
-
Using a physical test environment
When you test by using a physical environment, you must replicate your proposed production server farm environment as closely as possible. If you simplify the number of front-end web servers, application servers, or database servers too much, you will not have an accurate estimate of how long the upgrade process will take. You may not account for complications that arise from interactions between servers in the same role (such as SQL Server transactions). If you have multiple servers in a role in your proposed production farm, use at least two servers for that role in the test farm to test for such issues.
Distribution of servers in a physical test environment
- Make your test farm as similar as possible to your real farm — for example, hardware, software, and available space.
-
Identify and install customizations
To have an accurate test process, you must find all the customizations in your current environment and copy them to the test environment. For more information about the types of customizations that you have to identify, see Create a plan for current customizations during upgrade to SharePoint 2013.
- Use the Stsadm –o enumallwebs operation on all content databases in your SharePoint 2010 Products environment to identify specific customizations in subsites. This operation lists an ID for each site collection and subsite in your environment and the templates that the site relies on. For more information, see Enumallwebs: Stsadm operation.
- Use a tool such as WinDiff (a tool that is provided with most Microsoft operating systems) to compare your production environment servers with your test farm servers. You can use this tool to see which files exist on your servers and the differences between them.
- Check the web.config files for any changes, and look in the SafeControls element to find any custom controls.
- Use the SharePoint Diagnostics Tool (SPDiag) to find deployed solutions. For more information, see SharePoint Diagnostics Tool (SPDiag).
- Create a list of all customizations that you find. Identify the source of the customizations, if it is possible. For example, are there third-party add-ins or templates that were customized in-house? After you identify the source, you can then check for updated or upgraded versions of the customizations. Download the SharePoint 2013 Products Preview Upgrade Worksheet and record information about your environment, based on your research on your customizations.
Tip:
Whom do you contact about customizations that you did not create?
- Contact Microsoft if you have problems with a template that you downloaded from the Microsoft website.
- Contact your third-party solution vendor if you have problems with a template or component that they supplied you for the earlier version. An upgraded version may be available.
After you identify all the customizations, copy them to the appropriate servers in your test farm. Ensure that the following customizations are deployed:
- Solutions – by default legacy solutions are deployed to the /14 directories. Use the CompatibilityLevel parameter when you install the solutions to deploy them to the /15 directories. For more information, see Install-SPSolution.
- Custom Master Pages
- Custom JavaScript
- Custom CSS files (including those for themes)
- Custom workflow actions (must be included in actions file)
- Confirm large list query throttling settings to make sure that large lists are displayed as expected.
When you test customizations, use the following guidance:
- Check for visual changes.
- Check for behavioral changes.
- Test in both 2010 and 2013 mode site collections.
-
Look for any language or resource loading issues.
This is an issue that can occur when customizations exist in 2010 mode and new customizations replace them in 2013 mode. Because there is only one global directory for language resources, there can be an issue loading the correct file. Make sure that replacement 2013 customizations include the 2010 resources so that the customizations can continue to work correctly in both modes.
- Validate that upgrade did not affect customizations. Ensure that customizations do not block site collection upgrade.
You can use the Test-SPContentDatabase Windows PowerShell cmdlet before you attach a database to SharePoint 2013 to determine whether any customizations are missing from the environment. Run this command for each database after you restore the databases to your database server but before you run the upgrade. Note that this cmdlet runs silently — it will not return any output unless there is an issue found.
- Use the Stsadm –o enumallwebs operation on all content databases in your SharePoint 2010 Products environment to identify specific customizations in subsites. This operation lists an ID for each site collection and subsite in your environment and the templates that the site relies on. For more information, see Enumallwebs: Stsadm operation.
-
Copy real data to the test environment and upgrade databases
You cannot achieve your testing goals unless you use your actual data. Use the Microsoft SQL Server backup and restore tools to create a copy of your content and services databases.
There is no better way to tell what may occur during upgrade than to perform the test on a copy of all the data. However, this might not always be a realistic option for initial testing. You can test in phases by testing one database at a time (if the databases are large) so that you can make sure that you test whatever is unique about that dataset. Or, you can assemble a subset of data from representative sites in your environment. If you want to first test by using a subset of your data, be sure that the subset has the following characteristics:
- The data subset contains sites that are typical of the sites that you support in your environment.
- The size and complexity of the data subset closely resembles the actual size and complexity of your environment.
Important:
Testing a subset of your data does not produce a valid benchmark for how long it will take to process the whole volume of data for your environment.
After you copy the data, take a first pass through the upgrade process to see what happens. This is just the preliminary round. Follow the steps in Attach databases and upgrade to SharePoint 2013 to try the database attach upgrade process.
When you test the upgrade process, make sure that you test services that are shared across farms. Consider all states, such as the following:
- A SharePoint Server 2010 farm connected to a SharePoint Server 2013 services farm.
- A SharePoint Server 2013 farm connected to a SharePoint Server 2013 services farm.
- Different version farms for different services.
Use the test environment to find any security, configuration, compatibility, and performance issues for service applications.
- The data subset contains sites that are typical of the sites that you support in your environment.
-
Review results after you upgrade databases
After your test upgrade has finished, you can review the results and revisit your plans. Look at the log files, look at the upgraded sites, and review your customizations. How did upgrade work for your environment? What did you discover? What do you have to rethink about the upgrade plan?
-
Review the log files
Review the upgrade log file and the upgrade error log file (generated when you run the upgrade). The upgrade log file (.log) and the upgrade error log file (.err) are located at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LOGS. The log files are named in the following format: Upgrade-YYYYMMDD-HHMMSS-SSS.log, where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds).
The format of the log files complies with the Unified Logging System (ULS) conventions. To review the log files to find and troubleshoot issues, start at the top of the files. Errors or warnings may be repeated if they occur for several site collections in the environment, or if they block the upgrade process completely. For example, if you cannot connect to the configuration database, the upgrade process will try (and fail) several times and these tries will be listed in the log file.
-
Review sites in 2010 mode
Verify that the site collections that were not upgraded work as expected in 2010 mode. Sites should look and behave as they did in SharePoint 2010 Products. Some changes are expected. For example, Office Web Apps and the web analytics features have changed in SharePoint Server 2013 and sites that used these features will be affected. For information about specific things to look for, see Review site collections upgraded to SharePoint 2013.
-
Run upgrade again, if it is necessary
If you have to, you can restart the upgrade process for a database by using the Upgrade-SPContentDatabase Windows PowerShell cmdlet. For more information about this cmdlet, see Upgrade-SPContentDatabase. For more information, see Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013.
-
-
Upgrade site collections and My Sites
After you have tested and validated upgrade for the content and services databases, you can test the upgrade process for site collections. Follow the steps in Upgrade site collections to SharePoint 2013 to test the site collection upgrade process. If you have My Sites in your environment, see Overview of the upgrade process to SharePoint 2013 for more information about the process of upgrading them.
Note:
Content about My Sites applies only to SharePoint Server 2013.
-
Review results after you upgrade site collections
Review upgraded sites visually to identify any issues that have to be addressed before you run the upgrade process on your production environment. For more information about specific things to look for, see Review site collections upgraded to SharePoint 2013.
Review the site collection upgrade log files to check for any issues, starting from the top down. Check the summary section near the end of the log file to see a count of issues and the actual upgrade status (if there is no status, that means that the upgrade process failed and site upgrade must be retried). The site collection log files are stored both in the site collection itself (in the _catalogs/Upgrade document library), and on the file system. The file system log file has more information if you want details about issues. The file system version of the site upgrade log file is located at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LOGS. The log files are named in the following format: SiteUpgrade-YYYYMMDD-HHMMSS-SSS.log, where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds).
-
Adjust your plans and test again
Repeat the testing process until you are sure that you have found all the issues that you may face and that you know how to deal with them. Your goal is to know what your plan is if it is 4:00 P.M. on Sunday, you have to be back online Monday morning, and it is not going well. Is there a point of no return? Test your fallback plan and make sure that it works before you begin your real upgrade.
-
Troubleshoot database upgrade issues in SharePoint 2013
Published: July 16, 2012
Summary: Learn how to address problems that may occur after you upgrade a database to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Even after you test the upgrade process to identify potential issues, you might experience unexpected issues during an upgrade from SharePoint 2010 Products to SharePoint 2013. If you experience issues after upgrade, the sooner you detect and fix them, the better the end-user experience will be.
This article includes a list of common issues and describes general principles to help you identify and address upgrade issues. After you identify and address the issues, you can resume upgrade. For more information about how to resume upgrade, see Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013.
-
General principles to identify issues
Check the upgrade status to see where upgrade stopped (if it did stop), and check log files to find errors or warnings. Next, address the issues that you find before you resume the upgrade.
-
First, check upgrade status and log files
Upgrade status indicators and log files indicate what went wrong during the upgrade process. We recommend that you carefully review all the errors that were logged in the upgrade log files. Warnings might not always indicate an issue, but you should review them all to determine whether any of them are likely to cause even more issues.
-
-
Review the Upgrade Status page in the SharePoint Central Administration website.
For more information about how to check upgrade status, see Verify database upgrades in SharePoint 2013.
- Review the following log files:
- The upgrade error log file and the upgrade log file (which contains more detailed information than the upgrade error log file).
-
ULS or trace log files.
These files are stored in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\15\LOGS folder and are named Servername_YYYYMMDD–MMSS.log.
-
The application event log file.
This file can be viewed by using the Event Viewer.
For more information about the upgrade log files, see Verify database upgrades in SharePoint 2013. For more information about the trace log file, see Trace Logs on MSDN.
-
Then, address issues in order
Some issues have more effect than others. For example, a missing server-side file can cause many seemingly unrelated errors at the site level.
Address issues in the following order:
-
Missing server-side files or customizations, such as features or Web Parts.
Be sure to install all server-side customizations, such as features, Web Parts, and so on. Be sure to install customizations to the correct location in your new farm. For example, additional style sheets that you must have for SharePoint 2010 Products should be installed in the /14 path, not the new /15 path so that site collections that you have not upgraded can use them. Also, make sure that that you transfer all unique settings from the Web.config files for each web application to the new servers.
- Configuration issues in the server farm, web application, or service applications, such as managed paths or service applications that are not started.
- Additional issues that you discover on a site-by-site basis, starting with high-profile or very important sites.
As you identify and fix the top-level issues, you can try to run upgrade again to see whether any issues that occurred later in the upgrade process have also been fixed.
-
Common issues
Check to see whether any of the following issues cause an upgrade error or warning.
-
Q: I want to upgrade from a pre-release version of SharePoint 2013
-
A: Upgrade from a pre-release version of SharePoint 2013 to the release version of SharePoint 2013 is not supported.
Pre-release versions are intended for testing only and should not be used in production environments. Upgrading from one pre-release version to another is also not supported.
- A: Identify all server-side customizations and install them before you upgrade
One common error during upgrade is missing server-side files — either files that were installed with SharePoint 2010 Products or customized files. When you prepared for upgrade, you should have created an inventory of the server-side customizations (such as site definitions, templates, features, Web Parts, assemblies) that your sites required. Check this inventory to make sure that all the files that are needed for your customizations are installed in your new environment.
You can use the test-spcontentdatabaseWindows PowerShell cmdlet before you upgrade the database to identify missing files. You can also use the enumallwebs operation in Stsadm.exe to identify server-side customizations that are being used.
In the upgrade log files, you may see errors such as the following:
- ERROR Found Reference Count web(s) using missing web template Site Template Identifier (lcid: Site Template Language Code) in ContentDatabase Content Database Name.
- ERROR Found a missing feature Id = [Feature Identifier]
- WARNING File [Relative File Path] is referenced [Reference Count] times in the database, but is not installed on the current farm.
- WARNING WebPart class [Web Part Identifier] is referenced [Reference Count] times in the database, but is not installed on the current farm.
- WARNING Assembly [Assembly Path] is referenced in the database, but is not installed on the current farm.
- WARNING Feature could not be upgraded. Exception: Feature definition id ‘Feature Identifier’ could not be found.
If you can obtain a missing server-side file or dependency, install it, and then run upgrade again for the affected sites. If the file or dependency (such as a Web Part) was deprecated, you have to investigate whether you want to rebuild the site, page, or Web Part to use a different template, feature, or Web Part. If you can redo the customization by using dependencies that were not deprecated, you can run upgrade again for the affected sites. If you cannot remove the dependency, you cannot upgrade the site.
After you install the missing file or dependency, use the test-SPContentDatabase Windows PowerShell cmdlet on a test server to determine whether any other files for that database are missing. If you only run upgrade again, the error might not appear in the log files, even though it might still be occurring.
-
Q: The log file says that something is not right with my farm, web application, or service application configuration settings
- A: Verify your farm and web application settings.
- A: Create and start missing service applications
- A: Verify that managed paths (included paths) are configured correctly for each web application.
In the upgrade log files, you may see errors such as the following:
-
ERROR Template Template Id: SPSite Id=Site Id could not be accessed due to exception. Skipping SPWeb Id=Web Id for template upgrade. Exception: System.IO.FileNotFoundException: The site with the id Site Id could not be found.
This error indicates that a managed path is missing. Add the managed path for the site collection into the web application and restart upgrade for the content database that contains this site collection.
-
Q: I see errors and warnings during upgrade about connectivity or corruption
-
- A: Verify your power connections and connection to the network and to SQL Server. Loss of connectivity to data sources can cause errors. If your servers cannot connect to the databases, they cannot be upgraded.
-
A: Clean up orphaned sites, lists, and other database corruptions before you try upgrade again. For more information about how to clean up data issues, see Clean up an environment before an upgrade to SharePoint 2013.
In the upgrade log files, you may see errors such as the following:
- WARNING The orphaned sites could cause upgrade failures.
-
ERROR Database [Content Database Name] contains a site (Id = [Site Collection Identifier], Url = [Site Collection URL]) that is not found in the site map.
Fix any orphaned items or database corruptions, and then run upgrade again.
-
Q: I ran out of disk space
- WARNING The orphaned sites could cause upgrade failures.
-
A: Free some space, or increase the size of the transaction log file before you resume upgrade. If you run out of space (for example, for transaction log files on your database servers), upgrade cannot continue.
For more information, see Managing the Size of the Transaction Log File.
- A: Make sure that the web application is using the right authentication method.
A mismatch in authentication methods can cause problems when you upgrade. The following resources can help if you have a mismatch between authentication methods:
-
Classic-to-claims authentication
Make sure that the web applications that you created in SharePoint 2013 use the same authentication method that was used in SharePoint 2010 Products. Claims-based authentication is the default authentication method for web applications in SharePoint 2013. If the web application was using classic mode, you can either update it to claims before you upgrade the database, or create the web application in classic mode and then migrate it to claims. For more information about how to change to claims authentication in SharePoint 2010 Products, see Migrate from classic-mode to claims-based authentication in SharePoint 2013. For more information about how to create a web application that uses classic mode, and then migrating to claims, see Create web applications that use classic mode authentication in SharePoint 2013 and Migrate from classic-mode to claims-based authentication in SharePoint 2013
-
Forms-based authentication
Additional steps are necessary if you are upgrading an environment that uses forms-based authentication. Follow the steps in Configure forms-based authentication for a claims-based web application in SharePoint 2013 to upgrade forms-based authentication providers.
- A: If you receive an error about an unknown account, or if a database is not upgraded, check the permissions for the database. In particular, between instances of SQL Server, make sure that you verify that security is configured correctly. Check that the login accounts that you use have the appropriate fixed roles and permissions on the databases, and that they will still be valid if you upgrade across domains.
-
A: Make sure the account that you use to attach the databases is a member of the db_owner fixed database role for the databases that you want to upgrade.
-
Q: A database will not upgrade
-
-
A: Verify that the database is not set to read-only. You cannot upgrade a database that is set to read-only. Make sure that you set the databases to read-write before you attach and upgrade the databases.
-
Q: I changed a database name during restore, but I cannot find the files that have that name
-
-
A: When you rename a database at restore time, you must also rename the database and log file names in the file system (the MDF and LDF files) so that they match.
-
Q: I cannot back up the Search service application Administration database
-
-
A: Before you can back up the Search service application Administration database, you must stop the Search service on your SharePoint Server 2010 farm. To stop the Search service, on the original farm, on the Start menu, click Administrative Tools, and then click Services. Right-click SharePoint Server Search 14, and then click Stop. Be sure to start the service again after you back up the database.
-
Q: Trusted connections are not working for Excel Services after upgrade
-
-
A: You must manually create all trusted data connections for Excel Services after upgrade.
-
Q: My workflows are no longer associated correctly
-
- A: Verify that the Workflow Auto Cleanup timer job is turned off. If you had disabled the Workflow Auto Cleanup timer job in your SharePoint 2010 Products environment, make sure that you disable this timer job in the new environment also. If this timer job is enabled in the new environment and disabled in the SharePoint 2010 Products environment, you might lose workflow associations when you upgrade.
-
-
Troubleshoot site collection upgrade issues in SharePoint 2013
Updated: October 2, 2012
Summary: Learn how to address problems that may occur after you upgrade a site to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
When you upgrade a site collection to SharePoint 2013, errors can occasionally occur. This article helps you understand those errors and address them.
For more information about how to review UI issues in sites, see Review site collections upgraded to SharePoint 2013.
-
Check upgrade status and log files
Upgrade status indicators and log files should give you an indication of what went wrong during the upgrade process. We recommend that you carefully review all the errors in the upgrade log files. Warnings might not always indicate an issue, but you should review them all to determine whether any of them are likely to cause even more issues.
-
Review the upgrade status page for your site collection.
On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection upgrade. On the Site Collection Upgrade page, click Review Site Collection Upgrade Status.
- Review the site collection upgrade log files. You can review the site collection upgrade logs from the following locations:
- For site collection administrators: There are also log files for site collection upgrade stored inside the site collection itself, in the Maintenance Logs catalog at (http://<SiteName>/_catalogs/MaintenanceLogs/YYYYMMDD-HHMMSS-SSS.txt , where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds).
- For farm administrators: The site collection upgrade log file and the upgrade error log file are located at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LOGS. The logs are named in the following format: SiteUpgrade-YYYYMMDD-HHMMSS-SSS.log, where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds). These file system logs have more information if you want details about issues.
-
Common issues
Check to see whether any of the following issues are causing an upgrade error or warning or a problem in your site.
- A: Reset the page to the default version (that is, reghost it).
Making changes to the site UI can cause problems in site upgrades. If a page was customized to place a UI control in a non-standard location, you can reset the page to the default version to recover the control.
To reset the page, you can use the Reset to site definition link under Site Actions on the Site Settings page or use the Reset to Template command in SharePoint Designer.
-
Q: The view on a large list is not working any longer
- A: Create indexed columns, folders, or new views for large lists. You might have to add the indexed column to your existing views.
If a list is very large, and users use a view or perform a query that exceeds the limit or throttling threshold, the view or query will not be permitted. You can create indexed columns with filtered views, organize items into folders, set an item limit on the page for a large view, or use an external list. For more information about large list throttling and how to address issues with large lists, see Manage lists and libraries with many items on Office Online.
-
Q: I see an error about a duplicate content type name
- A: Rename content types or fields that conflict with default names.
Occasionally, custom elements (such as a content type) may have a name that conflicts with a name in the new version.
In the upgrade log files, you may see an error such as the following:
-
Failed to activate site collection features on site Site Url. Exception: A duplicate content type name “name” was found.
This error indicates that a third-party content type was added to the specified site in SharePoint Server 2010. During upgrade to SharePoint Server 2013 its name conflicted with the default content type by the same name. Rename the third-party content type in the specified site to a different name and run upgrade again. Note that either renaming or removing a content type can cause any customizations dependent on that content type to stop working.
-
Q: My site looks ugly, doesn’t behave as expected, or I see script errors
-
-
A: Either edit the page or reset the page to the default version, or remove or replace the custom files.
A problem with custom or inline JavaScript or CSS files can cause these issues.
-
Q: Custom content in my site disappeared or doesn’t work
-
-
A: Change the master page, or change the content so that it doesn’t require specific zones.
The master page might have different zone layouts and the content might no longer reference it correctly. As a last resort, you can also reset the page to the default version. However, if you reset the page, you might lose zone specific content.
-
Q: I receive an error that says a control or page cannot render
-
-
A: Do one of the following:
- If a Web Part was added that is not installed, contact the farm administrator to have it installed. If is a Web Part that is no longer available or not supported, then use the Web Part maintenance view to remove the Web Part from the page (remove, do not just close the Web Part).
-
If a page was directly edited, either edit it again to remove the control or Web Part or reset the page to the default version.
A Web Part or other control might have been added to the page that is not installed or is no longer supported. Either a Web Part was added to a zone or the page was directly edited to add a control or Web Part reference directly inline (possibly on a master page).
-
Q: I receive an error that I cannot create a subsite based on a site template because the site template uses the 2010 experience version and my site collection is in the 2013 experience version
- If a Web Part was added that is not installed, contact the farm administrator to have it installed. If is a Web Part that is no longer available or not supported, then use the Web Part maintenance view to remove the Web Part from the page (remove, do not just close the Web Part).
-
A: Recreate the site template in the 2013 experience.
To recreate the site template, create a new subsite based on the 2013 experience, customize it again to match the template that you had, and then save the customized subsite as a template (on the Site Settings page, click Save site as template).
- A: Reset the page to the default version (that is, reghost it).
-
Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013
Published: July 16, 2012
Summary: Learn how to restart a database-attach upgrade or a site collection upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
In some cases, you might have to restart upgrade to finish a database-attach upgrade from SharePoint 2010 Products to SharePoint 2013. For example: if a template or language pack is missing from the environment, or if you lose the connection to SQL Server, you will have to resolve the issue and then restart upgrade. You might also need to retry or restart a site collection upgrade if it was unable to complete.
Note:
One frequent cause of failures during upgrade is that the environment is missing customized features, solutions, or other elements. Be sure that any custom elements that you must have are installed on your front-end web servers before you start the upgrade process. You can use the test-spcontentdatabaseWindows PowerShell cmdlet to identify any custom elements that your sites might be using. For more information, see Identify and install customizations in the article “Use a trial upgrade to find potential issues.”
-
Restart upgrade for a database by using Windows PowerShell
If the upgrade ran into issues during the database-attach upgrade, you can restart the upgrade process for the database after you have addressed the issue by using a Windows PowerShell cmdlet.
To restart upgrade for a database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt (PS C:\>), type the following command:
upgrade-spcontentdatabase <Name>
Where:
-
Name is the database name that you want to upgrade.
You can also use the -id parameter and provide the database GUID instead of a database name. You can run the following cmdlet to find the GUID for a content database:
Get-SPContentDatabase -Identity <content_database_name>
For more information, see Upgrade-SPContentDatabase and Get-SPContentDatabase.
-
Restart upgrade for a site collection
If upgrade ran into issues during a site collection upgrade, you can restart the upgrade process for the site collection after you have addressed the issue. You can use either the Site Settings page or a Windows PowerShell cmdlet to restart upgrade for a site collection.
To restart upgrade for a site collection
- Verify that the user account that performs this procedure is a site collection administrator.
- On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection upgrade.
-
On the Site Collection Upgrade page, click Upgrade this Site Collection.
This option starts to upgrade your site collection. A box opens to verify that you want to start the process.
- Click I’m ready to start the actual upgrade.
Note:
The site collection health checks are run automatically in repair mode before the upgrade starts. The results from the health checks are included in the upgrade log for the site collection. If there is an error, you must address it before you can continue to upgrade.
The upgrade starts, and the Upgrade status page for the site collection is displayed. This page automatically updates while the upgrade is in progress and displays information about the process, such as the following:
- Errors or warnings
- When the upgrade started
-
Where you can find the upgrade log file
After the upgrade is complete, the Upgrade status page is displayed in the new user interface with the message, Upgrade Completed Successfully.
- Click Let’s see the new site to go to the home page.
Farm administrators can restart upgrade by using Windows PowerShell.
To restart upgrade for a site collection by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Upgrade-SPSite <http://site> -VersionUpgrade [-Unthrottled]
Where:
- <http://site> is the URL for the site collection.
- Add the option -Unthrottled option to skip the site collection upgrade queue and start the upgrade immediately.
For more information, see Upgrade-SPSite.
-
Upgrade databases from SharePoint 2010 to SharePoint 2013
Published: July 16, 2012
Summary: Find resources to help you perform the steps to upgrade databases from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
After you learn about the upgrade process, plan for your upgrade, and test your upgrade process by following the steps in the articles in Test and troubleshoot an upgrade to SharePoint 2013, you are ready to perform a database-attach upgrade to SharePoint 2013. Follow the steps in this section for both a trial upgrade and your actual upgrade for your production farm.
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about upgrading databases to SharePoint 2013.
-
Downloadable resources about upgrading databases
Download the following content for information about upgrading databases.
Content
Description
Describes the steps in the process for a database-attach upgrade
-
TechNet articles about upgrading databases
The following articles about how to upgrade databases are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
Content
Description
Use this checklist to make sure that you follow all necessary steps as you prepare for upgrade, perform the upgrade, and perform post-upgrade steps.
Follow these steps to configure a new SharePoint 2013 environment, and then attach and upgrade content and service application databases.
Verify that the upgrade for your databases has succeeded and that you are ready to begin to upgrade sites.
Migrate from classic-mode to claims-based authentication in SharePoint 2013
Convert SharePoint 2010 Products or SharePoint 2013classic-mode web applications to claims-based authentication or create new claims-based web applications in SharePoint 2013.
Configure forms-based authentication for a claims-based web application in SharePoint 2013
Learn how to configure forms-based authentication with an LDAP provider for a new SharePoint 2013 web application.
-
Additional resources about upgrade
The following resources about upgrade to SharePoint 2013 are available from other subject matter experts.
Content
Description
Upgrade and Migration Resource Center for SharePoint 2013 Products
Visit the Resource Center to find additional information about upgrades to SharePoint 2013.
Visit the Resource Center to learn about what‘s new in SharePoint 2013.
-
Checklist for database-attach upgrade (SharePoint 2013)
Updated: October 16, 2012
Summary: Use this checklist as you upgrade from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
This checklist helps you confirm that you follow all the steps that you must follow as you prepare for upgrade, perform the upgrade, and perform post-upgrade steps. This checklist applies only to upgrade of the content and service application databases. It does not apply to upgrade of My Sites or other site collections. For more information, see Upgrade site collections to SharePoint 2013.
In this article:
Some steps include notes about how long that step might take. These rough estimates only give you a relative idea of the duration of the step. To discover how much time each step will take for your environment, we recommend that you perform trial upgrades in a test environment. For more information, see Use a trial upgrade to SharePoint 2013 to find potential issues and Plan for performance during upgrade to SharePoint 2013.
Important:
The steps in this article apply to both SharePoint Foundation 2013 and SharePoint Server 2013, except for the steps about how to upgrade the service applications, which apply mostly to SharePoint Server 2013 (the Business Data Connectivity service application applies to both).
-
Prepare for upgrade
Follow these steps in order before you start an upgrade to SharePoint 2013:
-
Pre-upgrade steps
Step
Notes
[ ]
Create an inventory of server-side customizations in the environment
Create an inventory of the server-side customizations in your environment (solutions, features, Web Parts, event handlers, master pages, page layouts, CSS files, and so on). Record all customizations needed for your environment in the upgrade worksheet.
Detailed steps: Identify and install customizations in the “Use a trial upgrade to find potential issues” article.
Complete this step for the whole environment. Check each web server to make sure that you don’t miss any customizations. Keep the inventory up to date as you prepare for the upgrade.
[ ]
Clean up your environment
Before you begin to upgrade, make sure that your environment is functioning in a healthy state and that you clean up any content that you do not have to upgrade. Clean up any orphaned sites or data, address any large lists and large ACLs, remove extraneous document versions, and remove any unused templates, features and Web Parts.
Detailed steps: Clean up an environment before an upgrade to SharePoint 2013.
Complete this step one time for the whole environment.
This process might take days or weeks to finish.
[ ]
Test the upgrade process
Try out upgrade in a test environment to find any issues and determine how long your actual upgrade might take.
Detailed steps: Use a trial upgrade to SharePoint 2013 to find potential issues
Perform this step multiple times, until you are prepared to perform the actual upgrade.
-
-
Complete the database attach upgrade
Follow these steps in order while you upgrade the content and service application databases for your environment.
Detailed steps: Attach databases and upgrade to SharePoint 2013.
-
Prepare the new environment
Step
Notes
[ ]
Install and configure SharePoint 2013 and any language packs
Install the prerequisite software, and then install and configure SharePoint 2013.
Complete these steps on each server in your farm.
This step might take one hour or more, depending on the number of servers are in your environment.
[ ]
Configure service applications
Enable and configure the services that you need in your new environment.
Do not configure the following service applications – you will configure them while you upgrade their databases later in the process:
- Business Data Connectivity service
- Managed Metadata service
- PerformancePoint services
- Search
- Secure Store service
- User Profile service
Complete this step one time for the whole environment.
[ ]
Configure general farm settings
Reapply any general farm settings that you must have from your previous farm — such as blocked file types, e-mail setting, and quota settings — and add users or groups to the Farm Administrators group. Configure new settings such as usage and health data collection, diagnostic logging, and mobile accounts.
Important:
If you had disabled the Workflow Auto Cleanup timer job in your SharePoint 2013 environment, make sure that you disable this timer job in your new environment also. If this timer job is enabled in the new environment and disabled in the previous version environment, you might lose workflow associations when you upgrade. For more information about this timer job, see Disable preservation of workflow history (SharePoint Server 2010).
Complete this step one time for the whole environment.
-
Back up and restore databases
Step
Notes
[ ]
Record the passphrase for the Secure Store service application
The Secure Store service application uses a passphrase to encrypt information. You must record this passphrase so that you can use it in the new environment.
Complete this step one time for each Secure Store service application in the environment.
[ ]
Set the previous version databases to be read-only
If you want your original environment to remain available to users in a read-only state, set the databases to read-only before you back them up.
Complete this step for each content database in your environment.
Depending on your organization, you might need a database administrator to complete this step.
[ ]
Back up databases
Back up all the content databases and the following service application databases before you begin the database attach upgrade process:
- Business Data Connectivity
- Managed Metadata
- PerformancePoint
- Search Administration
- Secure Store
- User Profile: Profile, Social, and Sync databases
Complete this step for each content database and supported service application database in your environment.
This step can take an hour, several hours, or longer, depending on your dataset and your environment.
Depending on your organization, you might need a database administrator to complete this step.
[ ]
Export the encryption key for the User Profile service application
The User Profile Service service application requires an encryption key that is stored separately from the database and is needed if you want to upgrade the User Profile Sync database.
Complete this step one time for each User Profile service application in the environment.
[ ]
Restore a backup copy of the databases
Restore the databases from the backup.
Complete this step for each content database and supported service application database in your environment.
This step can take an hour or longer, depending on your dataset and your environment.
Depending on your organization, you might need a database administrator to complete this step.
[ ]
Set the restored databases to be read-write
Before you can attach and upgrade the databases that you copied to the new environment, you must set them to read-write.
Complete this step for each content database and supported service application database in your environment.
Depending on your organization, you might need a database administrator to complete this step.
-
Upgrade service application databases
Step
Notes
[ ]
Start the service application instances
Start the following service instances from Central Administration:
- Business Data Connectivity service
- Managed Metadata service
- PerformancePoint services
- Secure Store service
- User Profile service
Start the instance of the Search service application by using Windows PowerShell.
Complete this step one time for the whole environment.
[ ]
Upgrade the Secure Store service application
Use Windows PowerShell to create the new service application and upgrade the database, create a proxy and add it to the default proxy group, and then restore the passphrase from the previous environment.
Complete this step one time for each Secure Store service application in the previous environment.
[ ]
Upgrade the Business Data Connectivity service application
Use Windows PowerShell to create the new service application and upgrade the database. You do not have to create a proxy for the Business Data Connectivity service application.
Note:
The Business Data Connectivity service application is available in both SharePoint Foundation 2013 and SharePoint Server 2013.
Complete this step one time for each Business Data Connectivity service service application in the previous environment.
[ ]
Upgrade the Managed Metadata service application
Use Windows PowerShell to create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group. You must upgrade the Managed Metadata service application before you can upgrade the User Profile service application.
Complete this step one time for each Managed Metadata service application in the previous environment.
[ ]
Upgrade the User Profile service application
Use Windows PowerShell to create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group. After you have created the User Profile service application, you must import the Microsoft Identity Integration Server Key (MIIS) encryption key. Finally, you can start the User Profile Synchronization service.
Complete this step one time for each User Profile service application in the previous environment.
[ ]
Upgrade the PerformancePoint Services service application
Use Windows PowerShell to create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group.
Complete this step one time for each PerformancePoint Services service application in the previous environment.
[ ]
Upgrade the Search service application
Use Windows PowerShell to create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group.
Note:
This step applies to only SharePoint Server 2013. Although SharePoint Foundation 2013 includes search functionality, it is not the same Search service application that is in SharePoint Server 2013 and it cannot be upgraded.
Complete this step one time for each Search service application in the previous environment.
[ ]
Verify that all of the new proxies are in the default proxy group
Use the Get-SPServiceApplicationProxyGroup cmdlet to verify that all of the service application proxies are in the default proxy group.
Complete this step one time for the whole environment.
-
Create web applications
Step
Notes
[ ]
Create and configure web applications
Create a web application for each web application that existed in the old environment.
Complete this step one time for the whole environment.
[ ]
Reapply server-side customizations
Manually transfer all server-side customizations to your new farm. Refer to the inventory that you created in the upgrade worksheet to make sure that you install all components that your sites depend on to work correctly. When you install solutions, make sure that you add it to the appropriate path (/14 or /15). If you want a solution to be available to both paths, install it two times, and the second time use the CompatibilityLevel parameter when you install it, and it will be installed to the /15 path.
Make sure that you reapply customizations to all web servers in the farm.
[ ]
Verify custom components
Use the Test-SPContentDatabase Windows PowerShell cmdlet to verify that you have all the custom components that you need for that database.
Complete this step for each content database in your environment.
Running the cmdlet takes only a few minutes, but addressing issues might take longer.
-
Attach and upgrade content databases
Step
Notes
[ ]
Attach a content database to a web application
Attach the content database that contains the root site collection first. For My Sites, attach the content database that contains the My Site host before attaching databases that contain the My Sites.
You must perform this action from the command line. Use the Mount-SPContentDatabaseWindows PowerShell cmdlet.
Complete this step for one content database in your environment.
This step might take several minutes or several hours, depending on your dataset and hardware on the web servers, database servers, and storage subsystem.
[ ]
Verify upgrade for the first database
Verify that upgrade succeeded for the first database, and review the site to see whether there are any issues.
Detailed steps: Verify database upgrades in SharePoint 2013.
Complete this step for the content database that you just attached.
[ ]
Attach remaining databases
Attach and upgrade the remaining content databases in your environment. You must complete this action from the command line.
Complete this step for each of the remaining content databases in your environment.
This step might take several minutes or several hours, depending on your dataset, whether you are upgrading multiple databases in parallel, and the hardware on the web servers, database servers, and storage subsystem.
[ ]
Monitor upgrade progress
Use the Upgrade Status page in the SharePoint Central Administration website to monitor progress as your databases are upgraded.
Detailed steps: Verify database upgrades in SharePoint 2013.
Complete this step for each content database that you upgrade.
This step might take several minutes, an hour, several hours, or days, depending on your content.
[ ]
Verify upgrade for the remaining database
Verify that upgrade succeeded for the remaining databases.
Detailed steps: Verify database upgrades in SharePoint 2013.
Complete this step for each of the remaining content databases in your environment.
This step might take several minutes, an hour, several hours, or days, depending on your content.
-
-
Complete post-upgrade steps
Follow these steps in order after you perform a database-attach upgrade.
-
Post upgrade steps for database attach upgrade
Step
Notes
[ ]
Verify that site collections are working as expecting in 2010 mode
Review the site collections and make sure that they work in 2010 mode before you begin to upgrade any site collections. You can use a similar review list as the one provided for upgraded sites in Checklists for reviewing upgraded sites
Complete this step one time for your whole environment.
[ ]
Migrate user accounts to claims authentication, if it is necessary
By default, new web applications in SharePoint 2013 use claims authentication. If you were using classic authentication in the previous environment, you must migrate the users to claims authentication. For more information, see Migrate from classic-mode to claims-based authentication in SharePoint 2013.
Complete this step one time for every web application that has changed authentication methods.
[ ]
Update links that are used in any upgraded InfoPath form templates
For a database-attach upgrade, you exported and imported all InfoPath form templates in your environment when you created the new environment. After upgrade, you can now update the links that are used in those upgraded form templates to point to the correct URLs by using a Windows PowerShell cmdlet.
For more information, see Configure InfoPath Forms Services (SharePoint Server 2010).
Complete this step one time for your whole environment.
[ ]
Configure your Search topology
The architecture for the Search service has changed for SharePoint Server 2013. Plan and configure your Search topology to suit your environment and the new architecture. For more information, see Scale search for performance and availability (SharePoint Server 2013) and Manage search topology (SharePoint Server 2013).
Complete this step one time for your whole environment.
[ ]
Start a full crawl
After all content is upgraded and all settings are configured, you can start a full search crawl of your content. For more information, see Start, pause, resume, or stop a crawl (SharePoint Server 2013).
Complete this step one time for your whole environment.
A full crawl can take several hours or days to complete, depending on how much content is in your environment.
[ ]
Back up your farm
Back up your farm so that you have a current backup of your upgraded environment before you start to upgrade site collections. For more information, see Back up a farm in SharePoint 2013.
Complete this step one time for your whole environment.
-
-
Attach databases and upgrade to SharePoint 2013
Updated: October 16, 2012
Summary: Learn how to upgrade content and service application databases from SharePoint 2010 Products to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
When you use the database-attach upgrade approach to upgrade from SharePoint 2010 Products to SharePoint 2013, you upgrade the content databases and several service application databases. You do not upgrade the configuration database for your farm.
This article describes how to set up your new environment and perform a database-attach upgrade. This article does not provide steps for how to upgrade a site collection. For steps to upgrade a site collection, see Upgrade site collections to SharePoint 2013.
For an overview of the whole upgrade process, see Overview of the upgrade process to SharePoint 2013. For an overview of how to upgrade services, see Services upgrade overview for SharePoint Server 2013.
Important:
Although this article applies to both SharePoint Foundation 2013 and SharePoint Server 2013, the sections about how to upgrade service applications apply to SharePoint Server 2013. (The exception is the section about how to upgrade the Business Data Connectivity service application which applies to SharePoint Foundation 2013 and SharePoint Server 2013).
This article is about 35 printed pages.
-
Before you begin
Before you start to upgrade, you must collect information and settings about your existing environment. You have to know what is in your SharePoint 2010 Products environment before you can start to build your SharePoint 2013 environment. Gather information such as the following:
- Alternate access mappings
- Authentication providers and authentication modes that are being used
- Quota templates
- Managed paths
- Self-service site management settings
- Incoming and outgoing e-mail settings
- Customizations
Before you create the new environment for a database-attach upgrade, review the following information about permissions, hardware requirements, and software requirements.
- Hardware and software requirements (SharePoint 2013)
- Initial deployment administrative and service accounts in SharePoint 2013
Before you attach and upgrade databases, review the following information about permissions, hardware requirements, and software requirements. Follow the specified steps to install or configure prerequisite software or to change settings.
- Ensure that the account that you use to attach the databases is a member of the db_owner fixed database role for the content databases that you want to upgrade.
-
Check for and repair all database consistency errors.
For more information, see Database maintenance for SharePoint Server 2010.
You also have to turn off or remove services or components in the SharePoint 2010 Products environment that could cause errors in the upgrade process. The following services or components should be removed or stopped before you back up your databases:
-
Web Analytics The architecture for the Web Analytics service application is different in SharePoint 2010 Products. The presence of SharePoint Server 2010 Web Analytics information in your content databases could cause an error during upgrade. Stop the Web Analytics service application before you back up the content databases. Features and Web Parts from Web Analytics in SharePoint Server 2010 will not exist in SharePoint Server 2013, even for a site collection in 2010 mode. Remove any Web Analytics Web Parts or features from SharePoint Server 2010site collections before upgrade.
For more information about this change to Web Analytics, see Changes from SharePoint 2010 to SharePoint 2013.
- PowerPoint Broadcast Sites The Office Web Apps have changed into a separate server product, Office Web Apps Server, which can serve multiple SharePoint farms for viewing and editing documents. Because of this change, PowerPoint Broadcast sites cannot be upgraded to SharePoint Server 2013. For more information about how to install and use Office Web Apps Server with SharePoint 2013, see Deploy Office Web Apps (Installed on SharePoint 2010 Products).
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Alternate access mappings
-
Install SharePoint 2013 in a new environment
Before you can upgrade your databases, you must use SharePoint 2013 to configure a new server or server farm. The first step in creating your new environment is to install SharePoint Server 2013 or SharePoint Foundation 2013 and configure your new server or server farm. You must do the following:
- Run the Microsoft SharePoint Products Preparation Tool to install all required software.
- Run Setup to install the product.
- Install all language packs that you want in your environment.
Note:
For more information about how to install available language packs, see Install or uninstall language packs for SharePoint 2013.
- Run the SharePoint Products Configuration Wizard to configure your server or servers.
Important:
Some service applications can be upgraded by using a service application database upgrade. If you want to upgrade these service applications by upgrading the service application databases, do not use the Farm Configuration Wizard to configure these service applications when you set up your new farm.
For step-by-step instructions for these tasks, see Install SharePoint 2013.
-
Configure service applications and farm settings
You must create the service applications on your new farm before you upgrade your content databases. The steps in Install SharePoint 2013 describe how to use the Farm Configuration Wizard to enable all service applications.
You can upgrade the following service applications by performing a services database upgrade:
- Business Data Connectivity service
- Managed Metadata service
- PerformancePoint services
- Search
- Secure Store service
- User Profile service
For an overview of how to upgrade these service applications, see Services upgrade overview for SharePoint Server 2013. The steps that you must follow to upgrade these service application databases are included in the Upgrade service application databases section.
The following services in SharePoint Server 2013 also require additional steps to enable and configure when you upgrade:
-
Excel Services
You can use the Farm Configuration Wizard to enable this service, but you must make sure that you create all trusted data connections again. For more information, see Configure Excel Services in SharePoint.
-
InfoPath Forms Service
This service is not part of the Farm Configuration Wizard. To use this service, use the Configure InfoPath Forms Services link on the General Application Settings page in Central Administration to configure it. To continue to use form templates from your previous environment, export all administrator-deployed form templates (.xsn files) and data connection files (.udcx files) from your SharePoint Server 2010 environment, and then import them to your new SharePoint Server 2013 environment. For more information, see Configure InfoPath Forms Services (SharePoint Server 2010)
-
Office Web Apps
Office Web Apps Server is a new stand-alone server product that delivers Office Web Apps functionality on your private network. You install and managed it separately from SharePoint Server 2013. It cannot be installed on the same server or virtual machine instance as SharePoint 2013. For more information, see Deploy Office Web Apps Server 2013.
The next step in creating the new environment is to apply general farm settings. You must manually reapply configuration settings from your SharePoint 2010 Products farm, such as the following:
-
Incoming and outgoing e-mail settings
For more information, see Configure incoming email for a SharePoint 2013 farm and Configure outgoing email for a SharePoint 2013 farm.
- All farm–level security and permission settings, such as adding user or group accounts to the Farm Administrators group
-
Blocked file types
For more information, see Manage blocked file types (SharePoint 2013).
And you must configure all new farm-level settings that you want to use, such as the following:
-
Usage and health data collection
For more information, see Configure usage and health data collection (SharePoint 2013).
-
Diagnostic logging
For more information, see Configure diagnostic logging (SharePoint 2013).
- Settings and schedules for timer jobs
Important:
If you had disabled the Workflow Auto Cleanup timer job in your SharePoint 2010 Products environment, make sure that you disable this timer job in your new environment also. If this timer job is enabled in the new environment and disabled in the SharePoint 2010 Products environment, you might lose workflow associations when you upgrade. For more information about this timer job, see Disable preservation of workflow history (SharePoint Server 2010).
You will create web applications later in the process, after you upgrade the service application databases. For more information, see Create web applications.
- Business Data Connectivity service
-
Record the passphrase for the Secure Store service application
The Secure Store service application uses a passphrase to encrypt information. You have to know what this passphrase is so that you can use it in the new environment. Otherwise, you will not have access to the information in the Secure Store. If you do not know the passphrase, you can refresh the key, and then back up the Secure Store database. For more information, see Working with encryption keys.
-
Set the previous version databases to be read-only
To maintain user access to your original environment, set the SharePoint 2010 Products databases to read-only before you back up the databases. Even if you don‘t want to maintain access over the long term, set the databases to read-only to make sure that you capture all the data in the backup so that you restore and upgrade the current state of the environment without allowing additional changes to be made. If the databases are set to read-only, users can continue to view content. However, they will be unable to add or change content.
Important:
You cannot upgrade a database that is set to read-only. Make sure that you set the databases to read-write before you attach and upgrade the databases. For more information about how to set the databases to read-write, see the steps in the Set the databases to read-write section.
To set a database to read-only in SQL Server 2005, SQL Server 2008, or SQL Server 2008 R2
- Verify that the user account that is performing this procedure is a member of the db_owner fixed database role for the databases.
- In SQL Server Management Studio, in Object Explorer, connect to an instance of the Database Engine, expand the server, and then expand Databases.
- Find the database that you want to configure to be read-only, right-click the database, and then click Properties.
- In the Database Properties dialog box, in the Select a page section, click Options.
- In the details pane, under Other options, in the State section, next to Database Read-Only, click the arrow, and then select True.
You can use Transact-SQL to configure the READ_ONLY database availability option. For more information about how to use the SET clause of the ALTER DATABASE statement, see Setting Database Options.
-
Back up the SharePoint 2010 Products databases by using SQL Server tools
You back up the databases in SQL Server Management Studio. A backup copy of the database guarantees that you have the data in a safe state if you must enable the original farm again and is required for a database-attach upgrade. Repeat the procedure for the following databases in the SharePoint 2010 Products server farm:
- All content databases (default database name: WSS_Content_ID
-
The following service application databases:
- Service application
Default database name
Business Data Connectivity
BDC_Service_DB_ID
Managed Metadata
Managed Metadata Service_ID
PerformancePoint
PerformancePoint Service Application_ID
Search Administration
Search_Service_Application_DB_ID
Secure Store
Secure_Store_Service_DB_ID
User Profile: Profile, Social, and Sync databases
User Profile Service Application_ProfileDB_ID
User Profile Service Application_SocialDB_ID
User Profile Service Application_SyncDB_ID
The Business Data Connectivity service application is available in both SharePoint Foundation 2010 and SharePoint Server 2010. The other service applications are available only in SharePoint Server 2010. Although SharePoint Foundation 2010 includes search functionality, it is not the same Search service application that is in SharePoint Server 2010 and it cannot be upgraded.
- Service application
You do not have to back up the configuration or admin content databases, because you recreated these databases when you set up the SharePoint 2013 server farm. Upgrading the configuration or admin content databases and the Central Administration site collection is not supported.
After you complete this procedure, you will have created backups of the read-only content databases.
To back up a database in SQL Server 2005, SQL Server 2008, or SQL Server 2008 R2
- All content databases (default database name: WSS_Content_ID
- Verify that the user account that is performing this procedure is a member of the db_owner fixed database role for the databases.
- In Management Studio, in Object Explorer, connect to an instance of the Database Engine, expand the server, and then expand Databases.
-
Right-click the database that you want to back up, point to Tasks, and then click Back Up.
The Back Up Database dialog box appears.
- In the Source area, in the Database box, verify the database name.
- In the Backup type box, select Full.
- Under Backup component, select Database.
- In the Backup set area, in the Name box, either accept the backup set name that is suggested or type a different name for the backup set.
- In the Destination area, specify the type of backup destination by selecting Disk or Tape, and then specify a destination. To create a different destination, click Add.
- Click OK to start the backup process.
Repeat the previous procedure to back up all the content and appropriate service application databases that SharePoint 2010 Products uses in your environment.
Important:
Before you can back up the Search service application Administration database, you must stop the Search service on your SharePoint Server 2010 farm. To stop the Search service, on the original farm, on the Start menu, click Administrative Tools, and then click Services. Right-click SharePoint Server Search 14, and then click Stop. Be sure to start the service again after you back up the database.
-
Export the encryption key for the User Profile service application
The User Profile Service service application requires an encryption key that is stored separately from the database and is needed if you want to upgrade the User Profile Sync database. You must export the Microsoft Identity Integration Server Key (MIIS) encryption key from your SharePoint Server 2010 environment. You will import this exported key to the SharePoint Server 2013 environment after you upgrade the User Profile service application databases. By default, the key is located on the server that is running SharePoint Server 2010 and that is hosting the Microsoft Forefront Identity Manager services in the following directory: <root directory drive>\Program Files\Microsoft Office Servers\14.0\Synchronization Service\Bin.
To export the encryption key for the User Profile service application
- Verify that you have the following memberships:
- Administrators group on the server on which you are running the command.
-
Open the Command Prompt window, and then change to the following folder:
%Program Files%\Microsoft Office Servers\14.0\Synchronization Service\Bin\
-
To export the key, type the following at the command prompt, and then press ENTER:
miiskmu.exe
- In the Microsoft Identity Integration Server Key Management Utility wizard, verify that Export key set is selected, and then click Next.
- In the Account Name box, type the account name for the farm administrator.
- In the Password box, type the password for the farm administrator.
- In the Domain box, type the domain that contains the farm administrator account, and then click Next.
-
In the Specify export file name and location box, type or click browse to select the path and file name to use for the exported key, and then click Next.
The key is exported as a file that has a .BIN file name extension.
-
Verify the information, and then click Finish.
A message appears indicating that the key was successfully exported.
- Click Close to close the Microsoft Identity Integration Server Key Management Utility.
For more information, see Back up a User Profile Service application (SharePoint Server 2010).
-
Restore a backup copy of the database
After you configure the new SharePoint 2013 server farm, you can restore the backup copies of the databases to SQL Server 2008 R2. Start with one database, and then verify that the restoration has worked before you restore the other databases.
Important:
Be sure to keep a copy of your original backups in reserve, just in case upgrade fails and you have to troubleshoot and try again.
To restore a backup copy of a database in SQL Server 2008 R2
- Verify that the user account that is performing this procedure is a member of the db_owner fixed database role for the databases.
- After you connect to the appropriate instance of the SQL Server 2008 Database Engine, in Object Explorer, expand the server name.
-
Right-click Databases, and then click Restore Database.
The Restore Database dialog box appears.
- In the Restore Database dialog box, on the General page, type the name of the database to be restored in the To database list.
Tip:
When you type the name for the restored database, you do not have to use the original name. If you want to change the database name from a name with a long GUID to a shorter, more friendly name, this is an opportunity to make that change. Be sure to also change the database and log file names in the file system (the MDF and LDF files) so that they match.
- In the To a point in time text box, keep the default (Most recent possible).
- To specify the source and location of the backup sets to restore, click From device, and then use the browse button to select the backup file.
- In the Specify Backup dialog box, in the Backup media box, be sure that File is selected.
- In the Backup location area, click Add.
- In the Locate Backup File dialog box, select the file that you want to restore, click OK, and then, in the Specify Backup dialog box, click OK.
- In the Restore Database dialog box, under Select the backup sets to restore grid, select the Restore check box next to the most recent full backup.
- In the Restore Database dialog box, on the Options page, under Restore options, select the Overwrite the existing database check box.
- Click OK to start the restore process.
-
Set the databases to read-write
You must also set the databases back to read-write on your SharePoint 2013 farm before you attach and upgrade them.
To set a database to read-write in SQL Server
- In SQL Server Management Studio, in Object Explorer, connect to an instance of the Database Engine, expand the server, and then expand Databases.
- Select the database that you want to configure to be read-write, right-click the database, and then click Properties.
- In the Database Properties dialog box, in the Select a page section, click Options.
- In the details pane, under Other options, in the State section, next to Database Read-Only, click the arrow, and then select False.
-
About upgrading the service application databases
To upgrade a service application database, you create a new service application and provide the name of the existing database to use for the new service application. As the service application is created, the database is upgraded. This process has several steps.
-
Start the service instances
The first step is to start service instances for the five service applications that you can upgrade: the Business Data Connectivity service, Managed Metadata Web Service, PerformancePoint Services service, Secure Store service, User Profile service, and Search service. Most of these service instances can be started from Central Administration. However the SharePoint Server Search service instance must be started by using Windows PowerShell.
-
Create the service applications and upgrade the databases
After you have started the service instances, the next step is to create the service applications and upgrade the databases. You must use Windows PowerShell to restore the service application databases.
-
Create proxies for the service applications
After you have upgraded the service application databases, you create the proxies for the service applications and add them to the default proxy group. You must create proxies for the following service applications:
- Managed Metadata service application
- Search service application
- Secure Store service application
- PerformancePoint Services service application
-
User Profile service application
The Business Data Connectivity service application automatically creates a proxy and assigns it to the default proxy group when you create the service application.
- Verify that the proxies are in the default group
The following sections provide procedures to complete these steps.
Note:
The Business Data Connectivity service application is available in both SharePoint Foundation 2013 and SharePoint Server 2013. The other service applications are available only in SharePoint Server 2013. Although SharePoint Foundation 2013 includes search functionality, it is not the same Search service application that is in SharePoint Server 2013 and it cannot be upgraded.
-
Start the service instances
The following procedures start the service instances.
To start service application instances from Central Administration
- Start SharePoint 2013 Central Administration.
-
For Windows Server 2008 R2:
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Central Administration.
If SharePoint 2013 Central Administration is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Central Administration.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- In SharePoint 2013 Central Administration, on the Application Management page, in the Service Applications section, click Manage Services on Server.
- Next to the Business Data Connectivity service, click Start.
- Next to the Managed Metadata Web Service, click Start.
- Next to the PerformancePoint Services service, click Start.
- Next to the Secure Store Service, click Start.
- Next to the User Profile Service, click Start.
The Search service instance must be started by using Windows PowerShell because you cannot start it from Central Administration unless a Search Service application already exists.
To start the Search service instance by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To start the Search service instance, at the Windows PowerShell command prompt, type the following commands and press ENTER after each one:
$SearchInst = Get-SPEnterpriseSearchServiceInstance
# Stores the identity for the Search service instance on this server as a variable
Start-SPServiceInstance $SearchInst
# Starts the service instance
For more information, see Get-SPEnterpriseSearchServiceInstance and Start-SPServiceInstance.
-
Upgrade the Secure Store service application
To upgrade the Secure Store service application, you create the new service application and upgrade the database, create a proxy and add it to the default proxy group, and then restore the passphrase from the previous environment.
To upgrade the Secure Store service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the Secure Store service application, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$sss = New-SPSecureStoreServiceApplication -Name ‘Secure Store‘ -ApplicationPool $applicationPool -DatabaseName ‘SecureStore_Upgrade_DB‘ -AuditingEnabled
Where:
- SecureStore is the name that you want to give the new Secure Store service application.
-
SecureStore_Upgrade_DB is the name of the service application database that you want to upgrade.
This command sets a variable, $sss, that you use when you create the proxy later.
For more information, see New-SPSecureStoreApplication.
After you create the Secure Store service application and upgrade the database, you have to refresh the encryption key. For information about how to refresh the encryption key, see Refresh the encryption key.
-
Type the following command to create a proxy for the Secure Store service application:
Windows PowerShell
New-SPSecureStoreServiceApplicationProxy -Name ProxyName -ServiceApplication $sss -DefaultProxyGroup
Where:
- ProxyName is the proxy name that you want to use.
-
$sss is the variable that you set earlier to identify the new Secure Store service application.
Tip:
If you do not use the variable $sss, then you must use an ID to identify the Secure Store service application instead of a name. If you have to find the ID, you can run the Get-SPServiceApplication cmdlet to return a list of all service application IDs.
-
DefaultProxyGroup adds the Secure Store service application proxy to the default proxy group for the local farm.
For more information, see New-SPSecureStoreServiceApplicationProxy.
-
Type the following command to restore the passphrase for the Secure Store service application:
Update-SPSecureStoreApplicationServerKey -Passphrase <Passphrase>
Where:
-
<Passphrase> is the Passphrase for the Secure Store service application from your previous environment.
For more information, see Update-SPSecureStoreApplicationServerKey.
-
Upgrade the Business Data Connectivity service application
To upgrade the Business Data Connectivity service application, you create the new service application and upgrade the database. You do not have to create a proxy for the Business Data Connectivity service application. The Business Data Connectivity service application automatically creates a proxy and assigns it to the default proxy group when you create the service application.
Note:
The Business Data Connectivity service application is available in both SharePoint Foundation 2013 and SharePoint Server 2013.
To upgrade the Business Data Connectivity service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the Business Data Connectivity service application, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
New-SPBusinessDataCatalogServiceApplication -Name ‘BDC Service‘ -ApplicationPool $applicationPool -DatabaseName ‘BDC_Service_DB‘
Where:
- BDC Service is the name that you want to give the new Business Data Connectivity service application.
-
BDC_Service_DB is name of the service application database that you want to upgrade.
For more information, see New-SPBusinessDataCatalogServiceApplication.
-
Upgrade the Managed Metadata service application
To upgrade the Managed Metadata service application, you create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group. You must upgrade the Managed Metadata service application before you can upgrade the User Profile service application.
To upgrade the Managed Metadata service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the Managed Metadata service application, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$mms = New-SPMetadataServiceApplication -Name ‘Managed Metadata Service Application‘ -ApplicationPool $applicationPool -DatabaseName ‘Managed Metadata Service_DB‘
Where:
- Managed Metadata Service Application is the name that you want to give the new Managed Metadata service application.
-
Managed Metadata Service_DB is name of the service application database that you want to upgrade.
This command sets a variable, $mms, that you use when you create the proxy later.
For more information, see New-SPMetadataServiceApplication.
-
At the Windows PowerShell command prompt, type the following command to create a proxy for the Managed Metadata service application:
Windows PowerShell
New-SPMetadataServiceApplicationProxy -Name ProxyName -ServiceApplication $mmd -DefaultProxyGroup
Where:
- ProxyName is the proxy name that you want to use.
- $mmd is the variable that you set earlier to identify the new Managed Metadata service application.
-
DefaultProxyGroup adds the Managed Metadata service application proxy to the default proxy group for the local farm.
For more information, see New-SPMetadataServiceApplicationProxy.
-
Upgrade the User Profile service application
To upgrade the User Profile service application, you create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group. After you have created the User Profile Service service application, you must import the Microsoft Identity Integration Server Key (MIIS) encryption key. Finally, you can start the User Profile Synchronization service.
Note:
You must upgrade the Managed Metadata service application before you can upgrade the User Profile service application.
To upgrade the User Profile service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the User Profile service application, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$upa = New-SPProfileServiceApplication -Name ‘User Profile Service Application‘ -ApplicationPool $applicationPool -ProfileDBName ‘User Profile Service Application_ProfileDB‘ -SocialDBName ‘User Profile Service Application_SocialDB‘
-ProfileSyncDBName ‘User Profile Service Application_SyncDB‘
Where:
- User Profile Service Application is the name that you want to give the new User Profile service application.
- User Profile Service Application_ProfileDB is name of the User Profile service application Profile database that you want to upgrade.
- User Profile Service Application_SocialDB is name of the User Profile service application Social database that you want to upgrade.
-
User Profile Service Application_SyncDB is name of the User Profile service application Sync database that you want to upgrade.
This command sets a variable, $upa, that you use when you create the proxy later.
For more information, see New-SPProfileServiceApplication.
-
Type the following command to create a proxy for the User Profile service application:
Windows PowerShell
New-SPProfileServiceApplicationProxy -Name ProxyName -ServiceApplication ServiceApplicationID -DefaultProxyGroup
Where:
- ProxyName is the proxy name that you want to use.
- $upa is the variable that you set earlier to identify the new User Profile service application.
-
ServiceApplicationID is ID of the User Profile service application that you created earlier.
Tip:
If you do not use the variable $upa, then you must use an ID to identify the User Profile service application instead of a name. If you have to find the ID, you can run the Get-SPServiceApplication cmdlet to return a list of all service application IDs.
-
DefaultProxyGroup adds the User Profile service application proxy to the default proxy group for the local farm.
For more information, see New-SPProfileServiceApplicationProxy.
After you have created the User Profile Service service application, you must import the Microsoft Identity Integration Server Key (MIIS) encryption key. Import this key to the following directory: <root directory drive>\Program Files\Microsoft Office Servers\15.0\Synchronization Service\Bin.
To import the encryption key for User Profile service application
- Verify that you have the following memberships:
- Administrators group on the server on which you are running the command.
-
Open the Command Prompt window, and then change to the following folder:
%Program Files%\Microsoft Office Servers\15.0\Synchronization Service\Bin\
-
To import the key, type the following at the command prompt, and then press ENTER:
miiskmu.exe /i Path {0E19E162-827E-4077-82D4-E6ABD531636E}
Where:
-
Path is the path and file name for the key that you want to import.
You might also have to enter a user name and password. These are the credentials for the farm administrator.
For more information, see Install a software update (SharePoint Server 2010).
After you have imported the encryption key, you can start the User Profile Synchronization service.
-
Start the User Profile Synchronization service
- Start SharePoint 2013 Central Administration.
-
For Windows Server 2008 R2:
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
- Click Start, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Central Administration.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Central Administration.
If SharePoint 2013 Central Administration is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Central Administration.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
- In Central Administration, on the System Settings page, under Servers click Manage services on Server.
- Next to the User Profile Synchronization Service, click Start.
- In the Select the User Profile Application section, select the User Profile service application that you upgraded.
- In the Service Account Name and Password section, type the account name and password to use for the User Profile Synchronization service.
-
Upgrade the PerformancePoint Services service application
To upgrade the PerformancePoint Services service application, you create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group.
To upgrade the PerformancePoint Services service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the PerformancePoint Services service application, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$pps = New-SPPerformancePointServiceApplication -Name ‘PerformancePoint Service‘ -ApplicationPool $applicationPool -DatabaseName ‘PerformancePoint Service Application_DB‘
Where:
- PerformancePoint Service is the name that you want to give the new PerformancePoint Services service application.
-
PerformancePoint Service Application_DB is name of the PerformancePoint Services service application database that you want to upgrade.
This command sets a variable, $pps, that you use when you create the proxy later.
For more information, see New-SPProfileServiceApplication.
-
Type the following command to create a proxy for the PerformancePoint Services service application:
Windows PowerShell
New-SPPerformancePointServiceApplicationProxy -Name ProxyName -ServiceApplication ServiceAplicationNameorID -Default
Where:
- ProxyName is the proxy name that you want to use.
- $pps is the variable that you set earlier to identify the new PerformancePoint Services service application.
-
Default adds the PerformancePoint Services service application proxy to the default proxy group for the local farm.
For more information, see New-SPPerformancePointServiceApplicationProxy.
-
Upgrade the Search service application
To upgrade the Search service application, you create the new service application and upgrade the database, and then create a proxy and add it to the default proxy group.
Note:
This section applies to only SharePoint Server 2013. Although SharePoint Foundation 2013 includes search functionality, it is not the same Search service application that is in SharePoint Server 2013 and it cannot be upgraded.
To upgrade the Search service application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
To store the application pool for a particular service application as a variable, at the Windows PowerShell command prompt, type the following command:
Windows PowerShell
$applicationPool = Get-SPServiceApplicationPool -Identity ‘SharePoint Web Services default‘
Where:
-
SharePoint Web Services default is the name of the service application pool that will contain the new service applications.
This cmdlet sets the service application pool as a variable that you can use again in the cmdlets that follow. If you have multiple application pools and have to use a different application pool for a particular service application, you can repeat this step to get the appropriate application pool before you create the service application.
-
To upgrade the Search service application, at the Windows PowerShell command prompt, type the following command:
$searchInst = Get-SPEnterpriseSearchServiceInstance -local
# Gets the Search service instance and sets a variable to use in the next commandRestore-SPEnterpriseSearchServiceApplication -Name ‘<SearchServiceApplicationName>‘ -applicationpool $applicationPool -databasename ‘<SearchServiceApplicationDBName>‘ -databaseserver <ServerName> -AdminSearchServiceInstance $searchInst
Where:
- SearchServiceApplicationName is the name of the Search service application.
- AppPoolName is the application pool name.
- SearchServiceApplicationDBName is the name of the Search service application Administration database that you want to upgrade.
-
AdminSearchServiceInstanceID is the ID for the Search Service application instance.
Note:
A Search service application upgrade might fail because of an issue that occurs during upgrade, such as network or SQL Server latency. If an error message appears during the Search service application upgrade, do the following:
- Delete the Search Administration database that you were trying to upgrade.
-
Using the backup copy that you made of the Search Administration database, repeat the following procedures in this article for the Search service application only:
- Upgrade the Search service application by typing the command again at the Windows PowerShell command prompt.
For more information, see Restore-SPEnterpriseSearchServiceApplication.
You must follow several steps to create the Search service application proxy and add it to the default proxy group. You must complete separate actions to find the ID for the Search service application, create the new proxy, get the proxy ID, and then add the proxy to the default proxy group.
-
Type the following command to get the ID for the Search service application and store it as a variable:
Windows PowerShell
$ssa = Get-SPEnterpriseSearchServiceApplication
For more information, see Get-SPEnterpriseSearchServiceApplication.
-
Type the following command to create a proxy for the Search service application:
Windows PowerShell
New-SPEnterpriseSearchServiceApplicationProxy -Name ProxyName -SearchApplication $ssa
Where:
- ProxyName is the proxy name that you want to use.
-
$ssa is the variable that you set earlier to identify the new Search service application.
For more information, see New-SPEnterpriseSearchServiceApplicationProxy.
-
Type the following command to get the Search service application proxy ID for the proxy you just created and set it as the variable $ssap:
Windows PowerShell
$ssap = Get-SPEnterpriseSearchServiceApplicationProxy
For more information, see Get-SPEnterpriseSearchServiceApplicationProxy.
-
Type the following command to add the Search service application proxy to the default proxy group:
Windows PowerShell
Add-SPServiceApplicationProxyGroupMember –member $ssap -identity “ “
Where:
- $ssap is the variable that you set earlier to identify the ID for the proxy you just created for the Search service application.
-
You use an empty identity parameter (“ “) to add it to the default group.
For more information, see Add-SPServiceApplicationProxyGroupMember.
-
Verify that all of the new proxies are in the default proxy group
Use the following procedure to verify that the steps to create the proxies and add them to the default proxy group worked.
To verify that all of the new proxies are in the default proxy group by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following commands:
$pg = Get-SPServiceApplicationProxyGroup -Identity “ “
$pg.Proxies
Where:
- $pg is a variable you set to represent the default proxy group.
-
You use an empty identity parameter (“ “) to specify the default proxy group.
This returns a list of all proxies in the default proxy group, their display names, type names, and IDs.
For more information, see Get-SPServiceApplicationProxyGroup.
Now that the service applications are upgraded, you can start the process to upgrade the content databases. The first step in that process is to create the web applications that are needed for each content database.
-
Create web applications
Create a web application for each web application that existed in the SharePoint 2010 Products environment. For each web application, do the following:
-
Use the same URL and configure alternate-access mapping settings.
If you use a different URL, Office applications might not be redirected correctly to the new URLs and all bookmarks to the old URLs will not work.
-
Use the same authentication method.
For example, if you use Windows Classic authentication in your old environment, and you want to continue to use it, then you must create a web application that uses Windows Classic authentication. Because claims-based authentication is now the default option for SharePoint 2013, you must use Windows PowerShell to create a web application that uses Windows Classic authentication. For more information, see Create web applications that use classic mode authentication in SharePoint 2013 and Create claims-based web applications in SharePoint 2013.
Alternatively, you can migrate to claims authentication. For more information, see Migrate from classic-mode to claims-based authentication in SharePoint 2013.
- Recreate included paths.
- Recreate quota templates.
-
Configure email settings for the web application.
For more information, see Configure email integration for a SharePoint 2013 farm.
- Enable self-service site creation for any web application that used it in the previous environment. Recreate any self-service site creation settings.
- Create the managed path for the My Sites (/personal) on the web application that hosts My Sites. My Sites are available in SharePoint Server only.
- Recreate any web application policies or other web application settings that you had configured in the previous environment.
-
-
Reapply customizations
One frequent cause of failures during upgrade is that the new environment does not have customized features, solutions, or other elements. Make sure that all custom elements from the SharePoint 2010 Products environment are installed on your front-end web servers before you upgrade any content databases.
In this step, you manually transfer all customizations to your new farm. Make sure to install any components that your sites depend on to work correctly, such as the following:
- Custom site definitions
- Custom style sheets, such as cascading style sheets, and images
- Custom Web Parts
- Custom Web services
- Custom features and solutions
- Custom assemblies
-
Web.config changes (such as security)
Ensure that you transfer all unique settings from the Web.config files for each web application to the new servers.
- Administrator-approved form templates (.xsn files) and data connection files (.udcx files) for InfoPath. InfoPath is available in SharePoint Server 2010 only.
- Any other components or files on which your sites depend.
SharePoint 2013 can host sites in both SharePoint 2010 Products and SharePoint 2013 modes. The installation for SharePoint 2013 contains both SharePoint 2010 Products and SharePoint 2013 versions of many elements. The directories on the file system are duplicated in both the 14 and 15 paths, for example:
- Web Server Extensions/14/TEMPLATE/Features
- Web Server Extensions/15/TEMPLATE/Features
There are also two versions of the IIS support directories: _Layouts, _Layouts/15 and _ControlTemplates, _ControlTemplates/15.
Be sure to install customizations to the correct location in your new farm. For example, additional style sheets for SharePoint 2010 Products should be installed in the /14 path, not the new /15 path so that site collections that you haven‘t upgraded can use them. If you want a solution to be available to both paths, install it two times, and the second time use the CompatibilityLevel parameter when you install it, and it will be installed to the /15 path. For more information, see Install-SPSolution.
For more information about how to update customizations for use in SharePoint 2013, see Redeploying Customizations and Solutions in SharePoint Foundation 2010 and SharePoint Server 2010. For more information about how to deploy customizations to your environment, see Install and manage solutions for SharePoint 2013.
- Custom site definitions
-
Verify custom components
To make sure that you have identified all custom components for your environment, use the Stsadm -o enumallwebs operation in the SharePoint 2010 Products environment and use the includefeatures and includewebparts parameters. This operation can report the templates, features, Web Parts, and other custom elements that are used for each site. For more information about how to use the enumallwebs operation, see Enumallwebs: Stsadm operation (Office SharePoint Server) and Clean up an environment before an upgrade to SharePoint 2013.
You can also use the Get-SPWeb Windows PowerShell cmdlet in your SharePoint 2010 Products environment to see template that are associated with each site and then verify that the template is installed in your SharePoint 2013 environment. For more information about this operation, see Get-SPWeb.
Before you attach the content databases to the web applications, use the Test-SPContentDatabase Windows PowerShell cmdlet to verify that you have all the custom components that you must have for that database.
To verify custom components are available by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Test-SPContentDatabase -Name DatabaseName -WebApplication URL
Where:
- DatabaseName is the name of the database that you want to test.
- URL is the URL for the web application that will host the sites.
For more information, see Test-SPContentDatabase.
-
Attach a content database to a web application and upgrade the database
When you attach a content database, you upgrade the database and add the site collections in that database to the web application that you specify. However, for SharePoint 2013, the process does not upgrade the site collections.
When you attach a content database, include the root site for the web application in the first content database that you attach. In other words, before you continue, examine the root of the web application in the SharePoint 2010 Products server farm to determine the first site collection. After you attach the database that contains the root site, attach the other content databases for the web application in any order. You do not have to create any site collections to store the content before you attach the database. This process attaches the content databases and the site collections inside that database. Make sure that you do not add new site collections until you have restored all the content databases.
Tip:
Each site collection in a content database has a GUID that is registered in the configuration database and associated with the site collection. Therefore, you cannot add the same site collection two times to the farm, even in separate web applications. Although you can successfully attach the database in this situation, you will be unable to browse to the site collection.
If you must have a copy of a site collection in the same farm, first attach the database that contains the site collection to a separate farm, and then use the Backup-SPSite and Restore-SPSite Windows PowerShell cmdlets to copy the site collection to the other farm. The backup and restore process creates a new GUID for the site collection. For more information about these cmdlets, see Backup-SPSite and Restore-SPSite.
For My Sites, attach the content database that contains the My Site host before attaching databases that contain the My Sites.
By default, when you created the web applications in the new SharePoint 2013 environment, a content database was created for each web application. You can ignore these default databases until after you have attached your SharePoint 2010 Products databases, and then you can delete the default databases.
Important:
If you are moving the content databases across domains or forests or to another environment that has different service accounts, make sure that the permissions for the service accounts are still correct before you attach the databases.
You must use the Mount-SPContentDatabase cmdlet to attach a content database to a web application. Using the SharePoint Central Administration pages to attach a content database is not supported for upgrading.
Ensure that the account that you use to attach the databases is a member of the db_owner fixed database role for the content databases that you want to upgrade.
Note:
One frequent cause of failures during upgrade is that the environment is missing customized features, solutions, or other elements. Be sure that all custom elements from the SharePoint 2010 Productsenvironment are installed on your front-end web servers in the SharePoint 2013 environment before you start the upgrade process. Use the test-spcontentdatabase Windows PowerShell cmdlet to identify custom elements that your sites might be missing.
To attach a content database to a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Mount-SPContentDatabase -Name DatabaseName -DatabaseServer ServerName -WebApplication URL
Where:
- DatabaseName is the name of the database that you want to upgrade.
- ServerName is server on which the database is stored.
- URL is the URL for the web application that will host the sites.
For more information, see Mount-SPContentDatabase.
Tip:
To upgrade from SharePoint Foundation 2010 to SharePoint Server 2013, attach the SharePoint Foundation 2010 content databases directly to the SharePoint Server 2013 environment. Just follow the same steps in this article, only use the SharePoint Foundation 2010 databases and a SharePoint Server 2013 farm. The upgrade process will upgrade the version and the product at the same time.
-
Verification: Verify upgrade for the first database
After you attach a database, you can use the Upgrade Status page in Central Administration to check the status of upgrade on your databases. After the upgrade process is complete, you can review the upgrade log file to see whether upgrade produced issues. You can use a Windows PowerShell cmdlet to check the upgrade status for all the content databases. For more information about verifying and troubleshooting upgrade, see Verify database upgrades in SharePoint 2013 and Troubleshoot database upgrade issues in SharePoint 2013.
To view the Upgrade Status page
- Verify that the user account that is performing this procedure is a member of the db_owner fixed database role for the databases.
- In Central Administration, click Upgrade and Migration, and then click Check upgrade status.
To view the upgrade log file
-
The upgrade error log file and the upgrade log file are located at %COMMONPROGRAMFILES%\Microsoft Shared\web server extensions\15\LOGS. The upgrade log file contains more detailed information than the upgrade error log. Be sure to check the summary at the bottom of the log files for information about the overall status and a count of the warnings and errors in the file.
The logs are text files named in the following format:
- Upgrade-YYYYMMDD-HHMMSS-SSS-error.log
-
Upgrade-YYYYMMDD-HHMMSS-SSS.log
Where
- YYYYMMDD is the date
-
HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds)
An example for an upgrade error log is Upgrade-20120105-132126-374-error.log, and an example for an upgrade log is Upgrade-20120105-132126-374.log.
Note:
The format of the upgrade log for SharePoint 2013 is based on the same structure as ULS.
The upgrade log file includes the name of the content database being upgraded.
- Upgrade-YYYYMMDD-HHMMSS-SSS-error.log
To view upgrade status for all databases by using Windows PowerShell
- Verify that the user account that is performing this procedure is a member of the db_owner fixed database role for the databases.
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- Start the SharePoint 2013 Management Shell.
-
For Windows Server 2008 R2:
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
- On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, and then click SharePoint 2013 Management Shell.
-
For Windows Server 2012:
-
On the Start screen, click SharePoint 2013 Management Shell.
If SharePoint 2013 Management Shell is not on the Start screen:
-
Right-click Computer, click All apps, and then click SharePoint 2013 Management Shell.
For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.
-
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Get-SPContentDatabase | ft Name, NeedsUpgradeIncludeChildren
This cmdlet returns a table-style list of databases in your farm and indicates whether the database needs an upgrade to SharePoint 2013.
-
Attach the remaining databases
After you restore the first content database and verify success, you can continue to restore and upgrade other databases. You can perform parallel database attach upgrades to upgrade more than one database at a time. Use separate Command Prompt windows to run multiple upgrades. It is recommended that you separate the start time for each new database upgrade session by several minutes to prevent issues with temporary locks set for the web application during attachment. Otherwise you might receive an error on the upgrade session. The wait time to clear temporary locks varies depending on the number of site collections, or the speed of the database server hardware.
-
Verification: Verify upgrade for additional databases
After you upgrade all additional databases, view the Upgrade Status page to monitor progress and verify that the upgrade process is complete. Review the log file to identify any other issues.
-
Next steps
After you upgrade the databases, you might want to perform additional steps to make sure that your farm is ready for use. For example:
-
Verify that site collections are working as expecting in 2010 mode.
Visually review site collections. You can use a similar review list as the one provided for upgraded sites in Checklists for reviewing upgraded sites.
-
Migrate user accounts to claims authentication, if it is necessary.
By default, new web applications in SharePoint 2013 use claims authentication. If you were using classic authentication in the previous environment, you must migrate the users to claims authentication. For more information, see Migrate from classic-mode to claims-based authentication in SharePoint 2013.
-
Update links that are used in any upgraded InfoPath form templates.
For a database-attach upgrade, you exported and imported all InfoPath form templates in your environment when you created the new environment. After upgrade, you can now update the links that are used in those upgraded form templates to point to the correct URLs by using a Windows PowerShell cmdlet.
For more information, see Configure InfoPath Forms Services (SharePoint Server 2010).
InfoPath is available in SharePoint Server only.
-
Configure your Search topology
The architecture for the Search service has changed for SharePoint Server 2013. Plan and configure your Search topology to suit your environment and the new architecture. For more information, see Scale search for performance and availability (SharePoint Server 2013) and Manage search topology (SharePoint Server 2013).
-
Perform a full crawl
For more information, see Start, pause, resume, or stop a crawl (SharePoint Server 2013).
-
Back up your farm
For more information, see Back up a farm in SharePoint 2013.
Although SharePoint Foundation 2013 includes search functionality, it is not the same Search service application that is in SharePoint Server 2013. These steps apply only to SharePoint Server 2013.
After your farm is ready, you can enable access to users, and then start to upgrade site collections. For information about how to upgrade site collections, see Upgrade site collections to SharePoint 2013.
-
-
Verify database upgrades in SharePoint 2013
Published: July 16, 2012
Summary: Learn how to verify when a database-attach upgrade to SharePoint 2013 has finished, and identify any problems that may have occurred.
After you upgrade databases to SharePoint 2013, you must verify that the content was successfully upgraded to the new version. You can verify the status of the database-attach upgrade (is it still in progress, or has it been completed successfully or with errors or failures?) to see whether issues remain for you to address. When you follow these steps as part of a trial upgrade, you can use them to identify customizations that have to be reworked before you attempt to upgrade your production environment. When you upgrade your production environment, it is even more important that you know whether the upgrade has completed and what issues remain to be addressed.
In some cases, you might have to restart upgrade to finish upgrading your databases. For more information about how to restart upgrade, see Restart a database-attach upgrade or a site collection upgrade to SharePoint 2013. For information about how to restart a site collection upgrade, see Manage site collection upgrades to SharePoint 2013.
-
Verify upgrade status for databases
You can use the following methods to verify upgrade:
-
Use the Upgrade Status page in Central Administration
This page lists all farm, service, or content database upgrades and their statuses. This includes a count of errors or warnings.
-
Review the log files to look for errors or warnings
If upgrade was not successfully completed, you can view the log files to find the issues, address them, and then restart the upgrade process.
-
Review the log files for database attach upgrade
-
To verify that upgrade has succeeded, you can review the following log and error files:
-
The upgrade log file and the upgrade error log file.
Review the upgrade log file and the upgrade error log file (generated when you run the upgrade). The upgrade log file and the upgrade error log file are located at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LOGS. The logs are named in the following format: Upgrade-YYYYMMDD-HHMMSS-SSS.log, where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds). The upgrade error log file combines all errors and warnings in a shorter file and is named Upgrade-YYYYMMDD-HHMMSS-SSS-error.log.
The format of the log files complies with the Unified Logging System (ULS) conventions. To review the log files to find and troubleshoot issues, start at the top of the files. Errors or warnings may be repeated if they occur for several site collections in the environment, or if they block the upgrade process completely. For example, if you cannot connect to the configuration database, the upgrade process will try (and fail) several times and these tries will be listed in the log file.
If you find blocking issues in the log file, you can resolve the issues and then restart upgrade to continue with the process.
-
Check upgrade status for databases
The Upgrade Status page lists the upgrade sessions and gives details about the status of each session — whether it succeeded or failed, and how many errors or warnings occurred for each server. The Upgrade Status page also includes information about the log and error files for the upgrade process and suggests remedies for issues that might have occurred.
To view upgrade status in SharePoint Central Administration
-
- Verify that you have the following administrative credentials:
- To use SharePoint Central Administration, you must be a member of the Farm Administrators group.
- On the Central Administration home page, in the Upgrade and Migration section, click Check upgrade status.
-
Validate the upgraded environment
After you determine whether upgrade was completed successfully, validate your environment. Review the following items:
-
Service applications
- Are they configured correctly?
- Are the service application proxies configured the way that you want?
- Do you have to create new connections between farms?
- Are they configured correctly?
-
Site collections
- Are sites that were not upgraded working as expected in 2010 mode?
- Are all features associated with the sites working?
- Are sites that were not upgraded working as expected in 2010 mode?
-
Search
- Run a crawl, and review the log files.
- Run search queries, and verify that the queries work as expected and provide appropriate results. Twenty-four hours later, view the query reports and look for issues.
- Search for people and profiles.
- Check any Search customizations to make sure that they work as expected.
- Run a crawl, and review the log files.
-
-
Migrate from classic-mode to claims-based authentication in SharePoint 2013
Published: July 16, 2012
Summary: Convert SharePoint 2010 Products or SharePoint 2013 classic-mode web applications to claims-based authentication or create new claims-based web applications in SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Claims-based authentication is an essential component to enable the advanced functionality of SharePoint 2013. To move classic-mode web applications from SharePoint 2010 Products to SharePoint 2013, you can convert them to claims-based web applications within SharePoint 2010 Products, and then migrate them to SharePoint 2013. The procedures in this article illustrate various supported scenarios.
The Windows PowerShellConvert-SPWebApplication cmdlet in SharePoint 2013 converts classic-mode web applications to claims-based web applications.
Warning:
After you convert a web application to claims-based authentication, you cannot revert it to classic-mode authentication.
-
Convert SharePoint 2010 Products classic-mode web applications to claims-based authentication in SharePoint 2010 Products and then upgrade to SharePoint 2013
In SharePoint 2010 Products, complete the following procedure to convert an existing web application to claims-based authentication. After you convert the web application to claims-based authentication, complete the additional step to migrate the web application to SharePoint 2013. To complete this procedure, you need the following information:
- The URL of the web application that you are converting: http://yourWebAppUrl
- A user account to set as a site administrator: yourDomain\yourUser
To convert a SharePoint 2010 Products web application to claims-based authentication
- The URL of the web application that you are converting: http://yourWebAppUrl
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
- You must read about_Execution_Policies (http://go.microsoft.com/fwlink/p/?LinkId=193050).
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Permissions and Add-SPShellAdmin.
-
From the Windows PowerShell command prompt, type the following to set the specified user account as an administrator for the site:
$WebAppName = “http://<yourWebAppUrl>“
$wa = get-SPWebApplication $WebAppName
$wa.UseClaimsAuthentication = $true
$wa.Update()
Where:
- <yourWebAppUrl> is the URL of the web application.
-
From the Windows PowerShell command prompt, type the following to configure the policy to enable the user to have full access:
Windows PowerShell
$account = “yourDomain\yourUser”
$account = (New-SPClaimsPrincipal -identity $account -identitytype 1).ToEncodedString()
$wa = get-SPWebApplication $WebAppName
$zp = $wa.ZonePolicies(“Default”)
$p = $zp.Add($account,”PSPolicy”)
$fc=$wa.PolicyRoles.GetSpecialRole(“FullControl”)
$p.PolicyRoleBindings.Add($fc)
$wa.Update()
For more information, see Get-SPWebApplication.
-
From the Windows PowerShell command prompt, type the following to perform user migration:
$wa.MigrateUsers($true)
-
After user migration completes, type the following from the Windows PowerShell command prompt to perform provisioning:
Windows PowerShell
$wa.ProvisionGlobally()
For more information, see New-SPClaimsPrincipal.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
After you complete the previous procedures, you might experience one or more of the following issues:
- Users who submit valid credentials when accessing the migrated web application might be notified that they do not have permissions. If this occurs, the portalsuperuseraccount property and the portalsuperreaderaccount property of the web application were probably configured prior to migration. If this is the case, update the portalsuperuseraccount property and the portalsuperreaderaccount property to use the new claims-based account name. After migration, you can find the new claims-based account name in the web application policy for the migrated web application.
- If existing alerts are not invoked after migration, you might have to delete and recreate the alerts.
- If Search crawl does not function on the web application after migration, make sure that the Search crawl account lists the new converted account name. If the new converted account name is not listed, you must manually create a new policy for the crawl account.
To migrate a claims-based SharePoint 2010 Products web application to SharePoint 2013
- In SharePoint 2013, create a claims-based web application. For more information, see Create claims-based web applications in SharePoint 2013.
- Attach the two existing SharePoint 2010 Products content databases to the newly created SharePoint 2013 claims-based web application. For more information, see Attach or detach content databases in SharePoint 2013.
Note:
When you attach the SharePoint 2010 Products content databases to the SharePoint 2013 claims-based web application, the databases will be upgraded to the SharePoint 2013 database format. You have to verify that the content databases work correctly after you attach them.
-
Convert SharePoint 2010 Products classic-mode web applications to SharePoint 2013 claims-based web applications
In SharePoint 2013, complete the following procedure to convert an existing SharePoint 2010 Products classic-mode web application to a SharePoint 2013 web application that uses claims-based authentication.
To convert a SharePoint 2010 Products classic-mode web application to a SharePoint 2013 claims-based authentication
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
- You must read about_Execution_Policies (http://go.microsoft.com/fwlink/p/?LinkId=193050).
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Permissions and Add-SPShellAdmin.
- In the SharePoint 2013 environment, on the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
- Change to the directory where you saved the file.
-
At the Windows PowerShell command prompt, type the following command:
New-SPWebApplication –name “ClassicAuthApp” –Port 100 –ApplicationPool
“ClassicAuthAppPool” –ApplicationPoolAccount (Get-SPManagedAccount
“<domainname>\<user>“)
Where:
- <domainname>\<user> is the domain to which the server belongs and the name of the user account.
- Attach the two existing SharePoint 2010 Products content databases to the new SharePoint 2013 classic-mode web application. For more information, see Attach or detach content databases in SharePoint 2013.
Note:
When you attach the SharePoint 2010 Products content databases to the SharePoint 2013 classic-mode web application, the databases are upgraded to the SharePoint 2013 database format. You have to verify that the content databases work correctly after you have attached them.
-
From the Windows PowerShell command prompt, type the following:
Convert-SPWebApplication –Identity <yourWebAppUrl>
–To Claims
-RetainPermissions [ -Force]
Where:
-
<yourWebAppUrl> is the URL of the web application.
Note:
Convert-SPWebApplication converts the web application to claims-based authentication. You have to verify that the users can access the web application after you have converted it.
- If necessary, attach a third SharePoint 2010 Products content database to the new SharePoint 2013 classic-mode web application, and verify that the content database working correctly after you have attached it.
-
From the Windows PowerShell command prompt, type the following:
Convert-SPWebApplication –Identity yourWebAppUrl
–To Claims
-RetainPermissions [ -Force]
Verify that users can access the web application after you have converted it to claims-based authentication.
For more information, see New-SPWebApplication, Get-SPManagedAccount, and Convert-SPWebApplication.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Convert SharePoint 2013 classic-mode web applications to claims-based web applications
In SharePoint 2013, complete the following procedures to first create a classic-mode Web application, and then convert it to claims-based authentication.
To create a classic-mode Web application in SharePoint 2013
-
Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
- You must read about_Execution_Policies (http://go.microsoft.com/fwlink/p/?LinkId=193050).
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Permissions and Add-SPShellAdmin.
- securityadmin fixed server role on the SQL Server instance.
-
From the Windows PowerShell command prompt, type the following:
New-SPWebApplication –Name <Name>
–ApplicationPool <ApplicationPool>
-AuthenticationMethod <WindowsAuthType>
–ApplicationPoolAccount <ApplicationPoolAccount>
-Port <Port> -URL <URL>
Where:
- <Name> is the name of the new web application that uses classic-mode authentication.
- <ApplicationPool> is the name of the application pool.
- <WindowsAuthType> is either “NTLM“ or “Kerberos“. Kerberos is recommended.
- <ApplicationPoolAccount> is the user account that this application pool will run as.
- <Port> is the port on which the web application will be created in IIS.
-
<URL> is the public URL for the web application.
Note:
For more information, see New-SPWebApplication.
Note:
After you successfully create the web application, when you open the Central Administration page, you see a health rule warning that indicates that one or more web applications is enabled with classic authentication mode. This is a reflection of our recommendation to use claims-based authentication instead of classic mode authentication.
- <Name> is the name of the new web application that uses classic-mode authentication.
To convert a SharePoint 2013 classic-mode web application to claims-based authentication
-
From the Windows PowerShell command prompt, type the following:
Convert-SPWebApplication -Identity “http:// <servername>:port” -To Claims
–RetainPermissions [-Force]
Where:
- <servername> is the name of the server.
- <servername> is the name of the server.
Verify that users can access the web application after you have converted it to claims-based authentication.
For more information, see New-SPWebApplication, Get-SPManagedAccount, and Convert-SPWebApplication.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
-
Migrate SharePoint 2010 Products classic-mode web applications to SharePoint 2013 classic-mode web applications
In SharePoint 2013, complete the following procedure to create a classic-mode web application, and then migrate an existing SharePoint 2010 Products classic-mode Web application to SharePoint 2013.
To migrate a SharePoint 2010 Products classic-mode web application to SharePoint 2013
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running Windows PowerShell cmdlets.
- You must read about_Execution_Policies (http://go.microsoft.com/fwlink/p/?LinkId=193050).
-
Add memberships that are required beyond the minimums above.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 15 Products cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Permissions and Add-SPShellAdmin.
-
From the Windows PowerShell command prompt, type the following:
New-SPWebApplication –name “ClassicAuthApp” –Port 100 –ApplicationPool
“ClassicAuthAppPool” –ApplicationPoolAccount (Get-SPManagedAccount
“<domainname>\<user>“)
Where:
- <domainname>\<user> is the domain to which the server belongs and the name of the user account.
- Attach the two existing SharePoint 2010 Products content databases to the new SharePoint 2013 classic-mode web application. Verify that the content databases work correctly after you have attached them. For more information, see Attach or detach content databases in SharePoint 2013.
Note:
After migration has successfully completed, you might find a user who has not been migrated listed in the ULS log. Determine if the user still exists in your Active Directory domain, and then:
- If the user does not exist in your Active Directory domain, assign someone else as the site owner and designate the user as deleted in the UserInfo table. To designate a user as deleted, change the tp_deleted value in the UserInfo table for that user to 1.
- If the user does exist in your Active Directory domain, run the migration procedure again.
For more information, see New-SPWebApplication and Get-SPManagedAccount.
Note:
We recommend that you use Windows PowerShell when performing command-line administrative tasks. The Stsadm command-line tool has been deprecated, but is included to support compatibility with previous product versions.
-
Upgrade site collections to SharePoint 2013
Published: July 16, 2012
Summary: Find out how to upgrade a site collection to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
The following downloadable resources, articles on TechNet, video recordings, and related resources provide information about upgrading site collections to SharePoint 2013.
-
Downloadable resources how to upgrade site collections
Download the following content for information about how to upgrade site collections.
Content
Description
Describes the steps in the process for a database-attach upgrade.
-
TechNet articles about how to upgrade site collections
The following articles about how to upgrade site collections are available to view online. Writers update articles on a continuing basis as new information becomes available and as users provide feedback.
ContentDescription
Run the site collection health checks to verify your site is running well. Check each site before you upgrade to SharePoint 2013.
Site collection administrators can preview a copy of their sites in SharePoint 2013 mode, and then upgrade their sites.
Review site collections after you have upgraded them to SharePoint 2013.
Farm administrators manage the upgrade queue and throttling settings and upgrade site collections to SharePoint 2013.
-
Additional resources about how to upgrade to SharePoint 2013
The following resources about upgrade to SharePoint 2013 are available from other subject matter experts.
Content
Description
Upgrade and Migration Resource Center for SharePoint 2013 Products
Visit the Resource Center to find additional information about upgrades to SharePoint 2013.
Visit the Resource Center to learn about what‘s new in SharePoint 2013.
-
Run site collection health checks in SharePoint 2013
Published: July 16, 2012
Summary: Run the site collection health checks on each site to find issues before you upgrade to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
SharePoint 2013 includes a set of rules that you can run against a site collection to verify that it is working as expected. These rules are part of the site collection health checks. You can run the health checks from the Site Settings page or by using Windows PowerShell.
If you are upgrading a site collection to SharePoint 2013, the first step in the process is to run the health checks.
Upgrade step 1: Run site collection health checks
For a visual overview of the entire upgrade process, see Overview of the upgrade process to SharePoint 2013.
You run the health checks manually to prepare for an upgrade. In addition, the health checks are run automatically in repair mode when you start to upgrade a site collection. You can also run the health checks at any time to verify that a site is working as expected. The site collection pre-upgrade health checks examine a site collection and list potential upgrade issues, such as missing or unsupported elements. For example, the results itemize customized files so that you can identify the custom file and reset it to the default template in the site definition, if you want. After you run the checks, a report lists potential issues. The report also has information about how to address the issues.
The site collection health checker includes the following rules:
-
Site collection health check rules
Rule name
Description
Rule ID
Customized Files
This rule checks for any files that were customized (or unghosted) in the site collection or subsites. When run in repair mode, it can reset the page to the default (reghost the file).
cd839b0d-9707-4950-8fac-f306cb920f6c
Missing Galleries
This rule checks for all default galleries and reports if any are missing from the site collection or subsites.
ee967197-ccbe-4c00-88e4-e6fab81145e1
Missing Site Templates
This rule checks to make sure that the template the site is based on is available and reports if any elements are missing.
5258ccf5-e7d6-4df7-b8ae-12fcc0513ebd
Unsupported Language Pack References
This rule checks to make sure that the language packs that are used by the site collection exist and are referenced correctly by the site collection.
99c946f7-5751-417c-89d3-b9c8bb2d1f66
Unsupported MUI References
This rule checks to make sure that the multi-user interface elements that are used by the site collection exist and are referenced correctly by the site collection.
6da06aab-c539-4e0d-b111-b1da4408859a
-
-
Before you begin
This is the first step in upgrading a site collection. Before you upgrade a site collection, you must have already configured the environment that uses SharePoint 2013 and upgraded the databases. For more information about these steps, see Attach databases and upgrade to SharePoint 2013.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Run the site collection pre-upgrade health checks by using Site Settings
Site collection owners can run the site collection health checks from the Site Settings page in their site collections.
To run the pre-upgrade checks for a site collection
- Verify that the user account that performs this procedure is a site collection administrator.
- On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection health checks.
-
On the Run site collection health checks page, click Start checks.
A report lists all checked issues and issues that you should resolve.
- Resolve all issues, and then click Try it again to verify that you fixed them.
-
Run the site collection pre-upgrade health checks by using Windows PowerShell
Farm administrators can use the following Windows PowerShell cmdlets to run the site collection health checks and to repair issues: Test-SPSite Repair-SPSite.
To run the site collection health checks in test mode by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Either a site collection administrator or be granted full read (for test mode) for the web application by policy. For more information about permission policies for web applications, see Manage permission policies for a web application (SharePoint Server 2010).
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Test-SPSite -Identity <SiteURL> [-Rule <RuleID>]
Where:
- <SiteURL> is URL for the site collection you want to check.
- <RuleID> is ID for a specific rule that you want to run.
To run the site collection health checks in repair mode by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Either a site collection administrator or be granted full control (for repair mode) for the web application by policy. For more information about permission policies for web applications, see Manage permission policies for a web application (SharePoint Server 2010).
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Repair-SPSite -Identity <SiteURL> [-Rule <RuleID>]
Where:
- <SiteURL> is URL for the site collection you want to repair.
- <RuleID> is ID for a specific rule that you want to run.
-
Additional steps
If you are performing an upgrade to SharePoint 2013, you can start the site collection upgrade after you have addressed all issues from the health checks. You can create an upgrade evaluation site to try the new user interface for your site, or you can upgrade your site collection directly. To learn about how to create evaluation site collections or upgrade a site collection, see Upgrade a site collection to SharePoint 2013.
-
Upgrade a site collection to SharePoint 2013
Published: July 16, 2012
Summary: Learn how site collection administrators can preview a copy of their sites in SharePoint 2013 mode, and then upgrade their sites.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
After a server farm administrator has upgraded the databases, site collection administrators can upgrade individual site collections. When site collection administrators first browse to their sites after the database has been upgraded, a notification bar at the top of the site indicates that their sites can be upgraded. The choices are to Start now or Remind me later. Start now begins the site collection upgrade process.
To upgrade a site collection, site collection administrators complete the following steps:
- Run the site collection health checks to verify the site is ready to upgrade. For more information, see Run site collection health checks in SharePoint 2013.
- Create an upgrade evaluation site to preview the differences between versions. (Optional)
- Upgrade the site collection.
- Verify that upgrade was successful and the site works as expected. For more information, see Review site collections upgraded to SharePoint 2013.
This article discusses the second and third steps, and includes procedures for performing these tasks from Site Settings. For information about using Windows PowerShell cmdlets to upgrade sites from the command line, see Manage site collection upgrades to SharePoint 2013.
Upgrade step 2: Request evaluation site collection and Step 3: Upgrade the site
For a visual overview of the upgrade process, including site collection upgrade, see Overview of the upgrade process to SharePoint 2013. For more information about how farm administrators can control site collection upgrades, see Manage site collection upgrades to SharePoint 2013. For more conceptual information about site upgrade, including how to plan for upgrade, see Plan for site collection upgrades in SharePoint 2013.
Important:
If you upgrade from SharePoint Server 2010 to SharePoint Server 2013, there are special considerations for My Sites. (My Sites are not available in SharePoint Foundation 2013.) Make sure that you upgrade the My Site Host site collection before you allow users to access their individual My Sites in SharePoint Server 2013. This makes sure that the server software and database changes are complete so that users can upgrade their individual My Sites successfully.
A user can upgrade his or her My Site by following the steps to upgrade a site collection later in this article, or a farm administrator can upgrade My Sites by using Windows PowerShell.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
- Plan browser support (http://go.microsoft.com/fwlink/p/?LinkId=246502)
- Accessibility for SharePoint Products
- Accessibility features in SharePoint 15 Products (http://go.microsoft.com/fwlink/p/?LinkId=246501)
- Keyboard shortcuts (http://go.microsoft.com/fwlink/p/?LinkID=246504)
- Touch (http://go.microsoft.com/fwlink/p/?LinkId=246506)
-
Create an upgrade evaluation site (Optional)
As a site collection administrator, you can request a preview of your site collection; this is called an upgrade evaluation site collection. An upgrade evaluation site collection enables you to see your site‘s content in a new, separate copy of the site that is running on the SharePoint 2013. Unlike visual upgrade in SharePoint Server 2010, the upgrade evaluation site collection is a complete copy of the site collection, separate from the original. Actions that you take in the upgrade evaluation do not affect the original site.
When you request an evaluation site collection, the request is added to a Timer job (named “Create Upgrade Evaluation Site Collections”) which runs once a day. You will receive an e-mail message when the upgrade evaluation site is available. This might take up to 24 hours. The message includes a link to the evaluation site. Upgrade evaluation site collections are set to automatically expire (after 30 days by default). If yours expires before you have finished evaluating the changes, you can request another upgrade evaluation site collection.
To request an upgrade evaluation site collection
- Verify that the user account that performs this procedure is a site collection administrator.
- On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection upgrade.
-
On the Step up to SharePoint 2013 page, click Try a demo upgrade.
This option starts the process of generating an upgrade evaluation site collection.
-
In the Create Upgrade Evaluation Site Collection box, click Create Upgrade Evaluation Site Collection.
A box opens and informs you that a demo site request was received.
-
Click Close to close the box.
You will receive an e-mail message when the upgrade evaluation is available. The e-mail message will contain a link to the site collection. Review the site and confirm that your site collection will look and behave as expected in the new user interface.
After you have reviewed the upgrade evaluation and made any necessary changes in your original site based on your evaluation, you can upgrade your site collection.
Farm administrators can use Windows PowerShell to request an upgrade evaluation site collection. For more information, see Manage site collection upgrades to SharePoint 2013.
-
Upgrade a site collection
After you run the pre-upgrade checks and optionally review an upgrade evaluation site collection, you can upgrade your site collection to SharePoint 2013.
To upgrade a site collection
- Verify that the user account that performs this procedure is a site collection administrator.
- On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection upgrade.
-
On the Site Collection Upgrade page, click Upgrade this Site Collection.
This option starts the process of upgrading your site collection. A box opens to verify that you want to start the process.
- Click I’m ready to start the actual upgrade.
Note:
The site collection health checks are run automatically in repair mode before the upgrade starts. The results from the health checks are included in the upgrade log for the site collection. If there is an error, you must address it before you can continue to upgrade.
The upgrade starts, and the Upgrade status page for the site collection is displayed. This page automatically updates while the upgrade is in progress and displays information about the process, such as the following:
- Errors or warnings
- When the upgrade started
-
Where you can find the upgrade log file
After the upgrade is complete, the Upgrade status page is displayed in the new user interface with the message, Upgrade Completed Successfully.
- Click Let’s see the new site to go to the home page.
Farm administrators can use Windows PowerShell to upgrade a site collection. For more information, see Manage site collection upgrades to SharePoint 2013.
-
Verification
To verify that upgrade has succeeded, check the Upgrade status page for the site collection.
-
View upgrade status in Site Settings
Site collection administrators can view the Upgrade Status page in Site Settings to verify that upgrade has succeeded for a site collection.
To view upgrade status in Site Settings
-
- Verify that the user account that performs this procedure is a site collection administrator.
- On the Site Settings page for the site collection, in the Site Collection Administration section, click Site collection upgrade.
-
On the Site Collection Upgrade page, click Review Site Collection Upgrade Status.
The Upgrade Status page for the site collection is displayed.
Farm administrators can use Windows PowerShell to view site collection upgrade status. For more information, see Manage site collection upgrades to SharePoint 2013.
-
Additional steps
Next, review your upgraded site collection to be sure that everything is working as expected. For more information see Review site collections upgraded to SharePoint 2013.
-
Review site collections upgraded to SharePoint 2013
Published: July 16, 2012
Summary: Learn what to look for when you review site collections after you upgrade to SharePoint 2013 and find tips to address issues.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Use these steps in your test environment to identify any issues before you upgrade your production environment. And review your upgraded sites to fix any issues after you have upgraded a site collection.
When you perform tests before upgrading your environment:
- Begin by validating high-impact or high-profile sites, and then move on to lower-priority sites. As part of the planning process, you should have identified which sites are high-impact and high-profile and require immediate attention, and which can wait a bit longer.
- To verify basic functionality, create a new site collection by using a representative set of lists, libraries, Web Parts, and so on. Review the new site to make sure that the common, basic elements of your sites are working.
- If pages do not render, you can check the Site Settings page by going directly to the URL (http:// siteurl/_layouts/settings.aspx). If the Site Settings page works and the upgrade has succeeded, there might be issues with the master page or home page. If the Site Settings page does not work, go to the site collection upgrade log file to see whether you can get more information about the problem.
You can review the site collection upgrade logs from the following locations:
- For site collection administrators: There are also log files for site collection upgrade stored inside the site collection itself, in the Maintenance Logs catalog at (http://<SiteName>/_catalogs/MaintenanceLogs/YYYYMMDD-HHMMSS-SSS.txt , where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds).
- For farm administrators: The site collection upgrade log file and the upgrade error log file are located at %COMMONPROGRAMFILES%\Microsoft Shared\Web server extensions\15\LOGS. The logs are named in the following format: SiteUpgrade-YYYYMMDD-HHMMSS-SSS.log, where YYYYMMDD is the date and HHMMSS-SSS is the time (hours in 24-hour clock format, minutes, seconds, and milliseconds). These file system logs have more information if you want details about issues.
Use the following checklists to review your upgraded sites and look for issues for either trial upgrades or upgrades in a production environment.
- Begin by validating high-impact or high-profile sites, and then move on to lower-priority sites. As part of the planning process, you should have identified which sites are high-impact and high-profile and require immediate attention, and which can wait a bit longer.
-
Checklists for reviewing upgraded sites
-
Web Parts
The following table lists issues with Web Parts that can occur after upgrade and how to address them.
Tip:
To test your Web Parts quickly, you can build a new Web Part page that contains all the custom Web Parts before you test an upgrade, and then review the page for any missing or broken Web Parts after the trial upgrade.
What to check
What to do if there is a problem
Do all the Web Parts from your original site appear in your upgraded site?
If a Web Part zone exists in a customized (unghosted) page, but not in the site definition, the Web Parts from that Web Part zone may have been moved into the bottom zone on the page during the upgrade.
Either in Edit Mode for the page in the browser or in SharePoint Designer 2013, look for missing Web Parts in the bottom zone or other zones, or check whether the Web Parts were closed. For more information about how to work with Web Parts and Web Part zones in SharePoint Designer 2013, see the SharePoint Designer Help system.
Are there any broken Web Parts pages and are the Web Parts displayed correctly (in the correct zone, location, and size)?
Either in Edit Mode for the page in the browser or in SharePoint Designer 2013, drag the Web Part into the correct zone or modify the Web Part properties to correct any sizing or positioning problems.
Are there any extra or missing Web Parts?
Open the page either in Edit Mode for the page in the browser or in SharePoint Designer 2013. If you see additional Web Parts on your page, look for closed or inactive Web Parts on the original version of the page. Were the closed or inactive Web Parts opened by the upgrade process? If so, you can modify the Web Part properties to close these Web Parts.
If Web Parts are missing, look for errors in SharePoint Designer 2013 such as “Error Rendering Control” or “Missing Assembly.” These errors indicate that the Web Part was not installed or was configured incorrectly for the new environment and must be reinstalled or reconfigured.
Do the Web Parts work correctly?
Open the page either in Edit Mode for the page in the browser or in SharePoint Designer 2013, and look for errors that indicate that a component or service is missing. Make sure that any components or services that the Web Parts rely on exist in the upgraded site. Particularly for the database attach upgrade approach, you must make sure that you have installed all the components or services that you must have for your Web Parts, and that you have configured them correctly (for example, you have configured the Web.config Safe Controls list).
Update and redeploy any Web Parts that exist but no longer function correctly.
Are any Web Parts pages still checked out?
If you check out a page to make changes, make sure that you check in the page again.
Are your Excel Web Access Web Parts working correctly? Did you create your connections again correctly? Are external data sources still working?
Verify all connections and external data sources.
Tip:
If you have problems with a Web Part, append ?contents=1 to the end of the URL syntax (http:// siteurl/default.aspx?contents=1), and then press ENTER. This opens the Web Part Maintenance page where you can remove and repair the broken Web Part.
-
Large lists
By default, large list query throttling is turned on in SharePoint 2013. If a list is very large, and users use a view or perform a query that exceeds the limit or throttling threshold, the view or query will not be permitted. Check any large lists in your environment and have the site administrator or list owner address the issue. For example, they can create indexed columns with filtered views, organize items into folders, set an item limit on the page for a large view, or use an external list. For more information about large list throttling and how to address issues with large lists, see Manage lists and libraries with many items on Office Online.
-
Styles and appearance
The following table lists common issues with the style and appearance of your web site after upgrade and how to address them.
Tip:
Most of the issues in this section can be resolved by correcting the links to an item.
What to check
What to do if there is a problem
Are all the images on your pages displayed correctly?
Verify or fix the links to the images.
Are the appropriate cascading style sheet colors and styles used in the appropriate locations?
Verify or fix the links to the cascading style sheet file. Verify the link on the master page.
Theme choices are different in SharePoint 2013 – which theme do you want to use?
Your site’s home page, or other pages on your site, may look different after the site is upgraded. You may have to re-create or revise a theme and reapply it.
Do you have any JavaScript controls that are not working?
Verify or fix the links to the controls.
Are your pages displayed correctly in the browser?
Verify that any HTML on the page is in strict XHTML mode.
Are any script errors displayed on any pages?
Verify the scripts and links, and verify that any HTML is in strict XHTML mode.
-
Customized (unghosted) pages
Customized (also known as unghosted) pages are pages that were edited and are now unique versions of the pages for the site, instead of the default template pages. The following table lists issues with customized pages that can occur after upgrade and how to address them.
What to check
What to do if there is a problem
Are your customizations still in place?
Determine whether you have only one issue or a larger problem with the whole page.
If you added a brand-new page to your original site (for example, if you replaced Default.aspx with a different file instead of changing the existing Default.aspx file), the new page has no association with the site definition. Therefore, it might not resemble the other pages on the upgraded site — nor can it be reset to resemble them. If you want your customized page to have the same appearance and behavior as the other pages on your site, consider creating a brand-new page that is based on the site definition and then transferring your customizations to that new page.
Can you still access the editing controls on the pages?
If you customized the editing controls (for example, the Site Actions link or the Edit Page link in SharePoint 2010 Products), check whether they still appear. If they don’t appear, you can replace them with the editing controls of the new version by resetting the page to the default version.
Use the Reset to Template command in SharePoint Designer to reset the page to the default version (also known as reghosting). After you have restored the default page, you can then reapply your customizations in the browser by applying a different master page, or by reapplying the customizations in SharePoint Designer.
Are your customizations still appropriate in the new environment, or do you want to update to the new functionality and look?
If you want the new functionality and features, you must reset any customized pages to use the template. Resetting the page basically discards the customizations and attaches your page to the appropriate master page. Any customizations that you want can then be transferred to the master page instead of being stored in individual pages.
Use the Reset to Template command in SharePoint Designer to reset the page to the default version (that is, reghost it). After you have restored the default page, you can then reapply your customizations in the browser by applying a different master page, or by reapplying the customizations in SharePoint Designer.
Are any pages still checked out?
If you check out a page to make changes, make sure that you check in the page again.
-
-
Manage site collection upgrades to SharePoint 2013
Published: July 16, 2012
Summary: Learn how farm administrators can manage the upgrade queue and throttling settings and upgrade site collections to SharePoint 2013.
Applies to: SharePoint Foundation 2013 | SharePoint Server 2013
Even though site collection administrators can now upgrade their own sites to SharePoint 2013, server farm administrators can still control when and whether a site collection is upgraded by managing the upgrade queue. You can also view and manage the upgrade throttling settings for a web application or content database to manage your farm’s performance for site collection upgrades.
-
Before you begin to upgrade site collections to SharePoint 2013
Farm administrators can control settings for site collection upgrade, such as notifications, throttling, and the upgrade queue, and can upgrade site collections by using Windows PowerShell. Before you change these settings or upgrade a site collection, you should understand the settings and the implications for making changes. For more information about the settings for site collection upgrade, see Plan for site collection upgrades in SharePoint 2013. For information about how to upgrade a site collection from the Site Settings page, see Upgrade a site collection to SharePoint 2013.
Note:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:
-
Control upgrade notifications and self-service upgrade
When a site collection is available to upgrade, site collection administrators see a status bar on their sites indicating that they can upgrade them. They can choose to upgrade the site collection then or be reminded later. You can control settings for these notifications and control whether site collection administrators can upgrade their site collections. For more information about these properties, see Plan settings for upgrade notifications, self-service upgrade, and site collection creation.
To view the upgrade notification and self-service upgrade settings by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands to view the upgrade notification settings for a web application:
$wa=Get-SPWebApplication <URL>
$wa.UpgradeReminderDelay
$wa.UpgradeMaintenanceLink
Where:
-
<URL> is URL for the web application that you want to check.
This command returns the Upgrade reminder delay setting for the specified web application.
-
At the Windows PowerShell command prompt, type the following command to view the self-service upgrade setting for a site collection:
$site=Get-SPSite <URL>
$wa.AllowSelfServiceUpgrade
Where:
- <URL> is URL for the site collection that you want to affect.
For more information, see Get-SPWebApplication and Get-SPSite.
To change the upgrade notification and self-service upgrade settings for a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command to change the upgrade notification settings for a web application:
$wa=Get-SPWebApplication <URL>
$wa.UpgradeReminderDelay=<Value>
$wa.UpgradeMaintenanceLink=’<LinkURL>‘
Where:
- <URL> is URL for the web application that you want to affect.
- <Value> is the numeric value that you want to set for the delay (for example, 10 for 10 days).
- <LinkURL> is a link where the user can find more information.
-
At the Windows PowerShell command prompt, type the following command to change the self-service upgrade setting for a site collection:
$site=Get-SPSite <URL>
$wa.AllowSelfServiceUpgrade=<Value>
Where:
- <URL> is URL for the site collection that you want to affect.
- <Value> is either ‘true’ to allow site collection administrators to upgrade the site, or ‘false’ to not show them the notification and not allow them to upgrade.
For more information, see Get-SPWebApplication and Get-SPSite.
-
Control the compatibility range for site creation modes
You can control which mode (2010 or 2013, or both) can be used when a user creates a site collection. The CompatibilityRange property on a web application controls the site modes available for a web application. You can view or change the settings for CompatibilityRange by using Windows PowerShell.
To view the compatibility range for site creation modes for a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands to view the compatibility range settings for a web application:
$wa=Get-SPWebApplication <URL>
# Stores the web application at that URL as a variable
$wa.CompatibilityRange
# Returns the CompatibilityRange for the specified web application
Where:
-
<URL> is URL for the web application that you want to check.
This command returns the compatibility range for the specified web application. For example:
MaxCompatibilityLevel MinCompatibilityLevel DefaultCompatibilityLevel Singular
——————— ——————— ————————- ——–
15 14
15
False
-
At the Windows PowerShell command prompt, type the following commands to view the maximum, minimum, and default settings for a specific range:
[Microsoft.SharePoint.SPCompatibilityRange]::<RangeName>
Where:
-
RangeName is one of the following values: OldVersions, NewVersion, AllVersions.
This command returns the compatibility range for the specified value. For example, for NewVersion:
MaxCompatibilityLevel MinCompatibilityLevel DefaultCompatibilityLevel Singular
——————— ——————— ————————- ——–
15
15
15
True
For more information, see Get-SPWebApplication.
To change compatibility range for site creation modes for a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command to change the compatibility range settings to a specific range:
$wa=Get-SPWebApplication <URL>
# Stores the web application at that URL as a variable
$wa.CompatibilityRange = [Microsoft.SharePoint.SPCompatibilityRange]::<RangeName>
# Specifies which range to use
$wa.Update()
# Updates the CompatibilityRange setting to use only the range you specified
$wa.CompatibilityRange
# Returns the new CompatibilityRange for the web application
Where:
- <URL> is URL for the web application that you want to change.
- RangeName is one of the following values: OldVersions, NewVersion, AllVersions.
-
At the Windows PowerShell command prompt, type the following command to change the values for the CompatibilityRange manually:
$wa=Get-SPWebApplication <URL>
# Stores the web application at that URL as a variable
$range = New-Object Microsoft.SharePoint.SPCompatibilityRange(<Integer>,<Integer>)
# Creates a new compatibility range from <Integer> to <Integer>
$wa.CompatibilityRange = $range
# Specifies which range to use
$wa.Update()
#Updates the CompatibilityRange setting to use only the range you specified with $range
$wa.CompatibilityRange
# Returns the new CompatibilityRange for the web application
Where:
- <URL> is URL for the web application that you want to change.
-
Integer is a number to use as the minimum or maximum value. For example, (14,15) would set the MinCompatibilityLevel to 14 (2010) and the MaxCompatibilityLevel to 15 (2013). The DefaultCompatibilityLevel is automatically set to the lower of the MaxCompatibilityLevel and the current major version (for example, 15).
This command sets and then returns the range that you specified. For example:
MaxCompatibilityLevel MinCompatibilityLevel DefaultCompatibilityLevel Singular
——————— ——————— ————————- ——–
15 14
15
False
For more information, see Get-SPWebApplication.
-
Control the queue for upgrades of sites to SharePoint 2013
Every site that is set to upgrade is added to the queue, even if it is processed immediately. A site is removed from the queue after it is upgraded, or if it has encountered an error that must be addressed by a site collection or server administrator. If an unexpected failure occurs during the process (such as a power outage or service interruption), the site remains in the queue and the timer service will try the upgrade again automatically. Server farm administrators can manage the queue to remove a site from the queue, add a site to the queue, or upgrade a site manually.
Server farm administrators can manage the queue to do the following:
-
Determine site collections that are in the upgrade queue.
Each web application has its own upgrade queue. You can show the sites that are in the queue for a specific content database associated with that web application.
-
See all sites that are currently being upgraded.
You can view the queue and filter it to show only the sites that are currently being upgraded for a specific content database.
-
Add a site collection to the upgrade queue.
If you want to upgrade a site collection, you can add it to the queue.
-
Remove a site collection from the upgrade queue.
You can remove a site collection from the upgrade queue. Stop the timer job, remove the site from the queue, and then restart the timer job to resume upgrade for the remaining sites in the queue. You cannot remove a site collection from the queue if it is currently being upgraded.
The following procedure contains steps to view and manage the site collection upgrade queue.
To manage the upgrade queue by using Windows PowerShell
-
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
To view all site collections in the queue for a content database, at the Windows PowerShell command prompt, type the following command:
Get-SPSiteUpgradeSessionInfo -ContentDatabase <DatabaseName> -ShowInProgress -ShowCompleted -ShowFailed |ft
Where:
-
<DatabaseName> is name of the database that you want to check. You can also use the GUID for the database instead of the name.
For more information, see Get-SPSiteUpgradeSessionInfo.
-
To see all sites that are currently being upgraded, at the Windows PowerShell command prompt, type the following command:
Get-SPSiteUpgradeSessionInfo -ContentDatabase <DatabaseName> -ShowInProgress
Where:
-
<DatabaseName> is name of the database that you want to check. You can also use the GUID for the database instead of the name.
For more information, see Get-SPSiteUpgradeSessionInfo.
-
To see whether a particular site is in the queue, at the Windows PowerShell command prompt, type the following command:
Get-SPSiteUpgradeSessionInfo -Site <http://site>
Where:
-
<http://site> is URL for the site collection you want to add to the upgrade queue.
For more information, see Get-SPSiteUpgradeSessionInfo.
-
To add a site collection to the upgrade queue, at the Windows PowerShell command prompt, type the following command:
Upgrade-SPSite <http://site> -VersionUpgrade -QueueOnly
Where:
-
<http://site> is URL for the site collection you want to add to the upgrade queue.
For more information, see Upgrade-SPSite.
-
To remove a site collection from the upgrade queue, at the Windows PowerShell command prompt, type the following command:
Remove-SPSiteUpgradeSessionInfo -Identity <URL>
Where:
-
<URL> is URL for the site collection you want to add to the upgrade queue.
For more information, see Remove-SPSiteUpgradeSessionInfo.
-
Control site throttle settings for upgrade to SharePoint 2013
You can view and change the upgrade throttle settings for a content database and web application by viewing and setting the SPContentDatabase.ConcurrentSiteUpgradeSessionLimit and SPWebApplication.SiteUpgradeThrottleSettings properties. For descriptions of the properties that control throttle levels and the default values, see Plan for site collection upgrades in SharePoint 2013.
For more information about web application properties, see SPWebApplication Properties. For more information about content database properties, see SPContentDatabase Properties.
The following procedure provides steps to view upgrade throttling settings for a web application.
To view the upgrade throttle settings for a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
$wa = Get-SPWebApplication <URL>
$wa.SiteUpgradeThrottleSettings
Where:
-
<URL> is URL for the web application that you want to check.
This command returns the set of throttling settings for the specified web application. For example:
AppPoolConcurrentUpgradeSessionLimit : 5
UsageStorageLimit : 10
SubwebCountLimit : 10
Name :
TypeName : Microsoft.SharePoint.Administration.SPSiteUpgradeThrottleSettings
DisplayName :
Id : ca76dda0-7050-4c6b-a126-05917da39f8a
Status : Online
Parent : SPWebApplication Name=SharePoint – 80
Version : 8222
Properties : {}
Farm : SPFarm Name=SharePoint_ConfigUpgradedPersistedProperties : {}
For more information, see Get-SPWebApplication.
You can change the upgrade throttle settings for a web application. The following procedure provides steps to change the upgrade throttling settings for a web application.
To change the upgrade throttle settings for a web application by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
$wa=Get-SPWebApplication <URL>
$wa.SiteUpgradeThrottleSettings.AppPoolConcurrentUpgradeSessionLimit=<Value>
$wa.SiteUpgradeThrottleSettings.UsageStorageLimit=<Value>
$wa.SiteUpgradeThrottleSettings.SubwebCountLimit=<Value>
Where:
- <URL> is URL for the web applications that you want to affect.
-
Value is the numeric value that you want to set for that limit (for example, 8).
This command changes the throttling settings for a web application to the value that you supply.
For more information, see Set-SPWebApplication.
The following procedure provides steps to view upgrade throttling settings for a content database.
To view the throttle settings for a content database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
$db = Get-SPContentDatabase <DatabaseName>
# Stores the database name as a variable to use in the next command
$db.ConcurrentSiteUpgradeSessionLimit
# Returns the value for the limit for that database
Where:
-
<DatabaseName> is name of the database that you want to check. You can also use the GUID for the database instead of the name.
This command returns the set of throttling settings for the specified content database.
For more information, see Get-SPContentDatabase.
You can change the upgrade throttle settings for a content database. The following procedure provides steps to change the upgrade throttling settings for a content database.
To change the throttle settings for a content database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following commands:
$db = Set-SPContentDatabase <DatabaseName>
# Stores the database name as a variable to use in the next command
$db.ConcurrentSiteUpgradeSessionLimit=<value>
# Changes the limit to the value you specify.
Where:
- <DatabaseName> is name of the database that you want to affect. You can also use the GUID for the database instead of the name.
-
<value> is a numeric value to set the property to, such as 9.
This command changes the throttling settings for the specified content database to the value that you supply.
For more information, see Set-SPContentDatabase.
-
Create upgrade evaluation site collections by using Windows PowerShell
Site collection administrators can request a preview of their site collection. This preview site is called an upgrade evaluation site collection. Farm administrators can request an upgrade evaluation site collection by using Windows PowerShell.
To request an upgrade evaluation site collection by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
- Administrators group on the server on which you are running the Windows PowerShell cmdlets.
-
Either a site collection administrator or be granted full control (for repair mode) for the web application by policy. For more information about permission policies for web applications, see Manage permission policies for a Web application (SharePoint Server 2010).
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Request-SPUpgradeEvaluationSiteCollection -identity URL to site
Where:
- URL to site is the URL to a site collection in 2010 mode.
For more information, see Request-SPUpgradeEvaluationSite.
-
Upgrade site collections by using Windows PowerShell
You can upgrade a single site collection or all site collections in a specific database by using Windows PowerShell.
To upgrade a single site collection in a database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Upgrade-SPSite <http://site> -VersionUpgrade [-Unthrottled]
Where:
- <http://site> is the URL for the site collection.
- Add the option -Unthrottled option to skip the site collection upgrade queue and start the upgrade immediately.
This cmdlet upgrades the specific site collection to 2013 mode. For more information, see Upgrade-SPSite.
To upgrade all site collections in a database, use Windows PowerShell. However, because sites can continue to run in 2010 mode in the SharePoint 2013 environment, this is not a necessary procedure for most environments. If you do choose to upgrade all site collections immediately, site collection owners do not have an opportunity to use an upgrade evaluation site to preview the new user interface or change their original site before upgrading. We do not recommend that you upgrade all site collections immediately as part of your initial upgrade. However, you might want to upgrade all site collections after some time has passed and all customizations were verified in 2013 mode.
To upgrade all site collections in a database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Get-SPSite -ContentDatabase <DBName> -Limit All | Upgrade-SPSite -VersionUpgrade -QueueOnly
Where:
-
<DBName> is the name of the content database for which you want to upgrade all site collections.
The -QueueOnly parameter adds the site collections to the upgrade queue. This allows the timer job to perform parallel upgrades when it is possible and can save time. The sites are upgraded in the order in which they are added to the queue.
This cmdlet upgrades all site collections in the specific content database to 2013 mode.
-
View upgrade status by using Windows PowerShell
You can view upgrade status for all databases, for a single site collection, or for all site collections.
To view upgrade status for a single site collection by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Get-SPSiteUpgradeSessionInfo -Site <http://site>
Where:
-
<http://site> is the URL of the site collection.
This cmdlet returns the upgrade status for the specified site collection together with information about the upgrade session and a link to the log files for more information. For more information, see Get-SPSiteUpgradeSessionInfo.
-
Or, you can use the following command to view the information about a specific site collection upgrade:
$sc = Get-SPSite <http://site>
# Sets a variable for the site collection
$sc.CompatibilityLevel
# Returns the compatibility level for the site collection (either 14 or 15 for 2010 or 2013 mode)
$sc.UpgradeInfo
# Returns the upgrade information for the site collection
Where:
-
<http://site> is the URL of the site collection.
This command returns the compatibility level and upgrade information (such as a pointer to the log file) for the specified site collection. If the compatibility level is “15,” then it has been upgraded to 2013 mode. For more information, see Get-SPSite.
To view upgrade status for a single database by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Windows PowerShell
Get-SPSiteUpgradeSessionInfo –ContentDatabase <DatabaseName> -ShowInProgress -ShowCompleted -ShowFailed
Where:
-
<DatabaseName> is the name of the database that you want to check.
This cmdlet returns any site collections that have an upgrade in progress, completed, or failed and lists their status, plus a link to the log files for more information. You can use only one parameter to find only in progress, completed, or failed upgrades. For more information, see Get-SPSiteUpgradeSessionInfo.
To view upgrade status for all site collections by using Windows PowerShell
- Verify that you have the following memberships:
- securityadmin fixed server role on the SQL Server instance.
- db_owner fixed database role on all databases that are to be updated.
-
Administrators group on the server on which you are running the Windows PowerShell cmdlets.
An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint 2013 cmdlets.
Note:
If you do not have permissions, contact your Setup administrator or SQL Server administrator to request permissions. For additional information about Windows PowerShell permissions, see Add-SPShellAdmin.
- On the Start menu, click All Programs.
- Click Microsoft SharePoint 2013 Products.
- Click SharePoint 2013 Management Shell.
-
At the Windows PowerShell command prompt, type the following command:
Get-SPSite -Limit All
This cmdlet returns the URL for all site collections in the environment and the compatibility level (14 or 15) for each site collection.
Capacity Planning for Microsoft SharePoint 2010My Sites and Social Computing features
Capacity Planning for
Microsoft SharePoint 2010
My Sites and Social Computing features
This document is provided “as-is”. Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it.
Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.
© 2010 Microsoft Corporation. All rights reserved.
Capacity Planning for
Microsoft SharePoint 2010
My Sites and Social Computing features
Gaurav Doshi, Wenyu Cai
Microsoft Corporation
Applies to: Microsoft SharePoint Server 2010
Summary: This whitepaper provides guidance on performance and capacity planning for a My Sites and Social computing portal based on Microsoft® SharePoint® 2010. This documents covers:
- Test environment specifications, such as hardware, farm topology and configuration
- Test farm dataset
- Test data and recommendations for how to determine the hardware, topology, and configuration that you need to deploy a similar environment, and how to optimize your environment for appropriate capacity and performance characteristics.
Table of Contents
Executive Summary…………………………………………………………………………………………………………………………….4
Introduction5
Scenario5
Assumptions and prerequisites5
Glossary5
Overview7
Scaling approach7
Correlating lab environment with production environment7
Test notes8
Test setup…………………………………………………………………………………………………………………………………………..9
Hardware9
Software9
Topology and configuration10
Dataset and disk geometry11
Transactional Mix12
Results and analysis15
Comparison of all iterations15
Impact of people search crawl19
Analysis20
Recommendations23
Appendix24
Executive summary
Overall, here are the key findings from our testing for the My Sites and Social Computing Portal:
- The environment scaled up to eight front-end Web servers for one application server and one database server; increase in throughput was almost linear throughout. After eight front-end Web servers, there are no additional gains to be made in throughput by adding more front-end Web servers because the bottleneck at this point was the database server CPU utilization.
- Further scale can be achieved by separating the Content database and Services database on two separate database servers.
- We maxed out the 8x1x2 topology. At that point both front-end Web server and application server CPU utilization was the bottleneck. That leads us to believe that for the given hardware, dataset, and test workload, the maximum RPS possible is represented by Max Zone RPS for 8x1x2, which is about 1877.
- Looking at the trends, it seems possible to extract the same throughput with a healthy farm, if the bottlenecks on the front-end Web server and application server are addressed. The front-end Web server bottleneck can be addressed by adding more front-end Web servers, and the application server bottleneck can be addressed by using two computers to play the role of application server. We did not try it out in the lab though.
- Latency is not affected by throughput or hardware variations.
- If you have security trimming turned ON, one front-end Web server can support about 8-10 RPS of Outlook Social Connector traffic. This means, one front-end Web server can support about 28,000 to 36,000 employees using Outlook Social Connector all day. Thus, if you are rolling out Outlook Social Connector to 100,000 employees, you can support the traffic that is generated by three front-end Web servers. These values can vary depending on social tagging usage at your company. If you imagine your company to have less social tagging activity than what we used in the dataset for this testing effort, you might get better throughput per front-end Web server.
- The incremental people search crawl doesn’t have much effect on the farm’s throughput as long as the farm is maintained in a healthy state.
Introduction
Scenario
This document outlines the test methodology and results to provide guidance for the capacity planning of a social computing portal. A social computing portal is a Microsoft® SharePoint® 2010 deployment where each person in the company maintains a user profile, finds experts in the company, connects with other employees through newsfeeds and maintains a personal site for document storage and sharing. In addition to this traffic caused by social computing features, there is good amount of typical collaboration traffic caused by people uploading, sharing, viewing, and updating documents on their personal sites. We expect these results to help in designing a separate portal dedicated to My Sites and social features.
Different scenarios will have different requirements, so it is important to supplement this guidance with additional testing on your own hardware and in your own environment.
When you read this document, you will understand how to:
- Estimate the hardware required to support the scale you need to support: number of users, load, and the features enabled.
- Design your physical and logical topology for optimum reliability and efficiency. High Availability/Disaster Recovery are not covered in this document.
- Account for effect of ongoing people search crawl and profile sync on the RPS of a social computing portal-like deployment
Before you read this document, you should read the following:
- Capacity Planning and Sizing for Microsoft SharePoint 2010 Products and Technologies
- Office SharePoint Server 2010 Software Boundaries
- SharePoint Server 2010 Technical Case Study: Social Environment, available for download on TechNet
If you are interested in reading capacity planning guidance on typical collaboration scenarios, please read: SharePoint Server 2010 Capacity Lab Study: Enterprise Intranet Collaboration Solution
Assumptions and prerequisites
- There is no custom code running on the social computing portal deployment in this case. We cannot guarantee the behavior of custom code or third party solutions that are installed on top of your My Site and social computing portal.
- Authentication mode was NTLM
Glossary
There are some specialized terms you will encounter in this document. Here are a few key terms and their definitions.
- RPS: Requests per second. The number of requests received by a farm or server in one second. This is a common measurement of server and farm load.
Note that requests are different from page loads; each page contains several components, each of which creates one or more requests when the page is loaded. Therefore, one page load creates several requests. Typically, authentication checks and events that are consuming negligible resources are not counted in RPS measurements. -
Green Zone: This is the state at which the server can maintain the following set of criteria:
- The server-side latency for at least 75 percent of the requests is less than 0.5 second.
- All servers have a CPU utilization of less than 50 percent.
Note: Because this lab environment did not have an active search crawl running, the database server was kept at 40 percent CPU utilization or lower, to reserve 10 percent for the search crawl load. This assumes Microsoft SQL Server® Resource Governor is used in production to limit Search crawl load to 10 percent CPU. - Failure rate is less than 0.01 percent.
-
Max Zone: This is the state at which the server can maintain the following set of criteria:
- HTTP request throttling feature is enabled, but no 503 errors (Server Busy) are returned.
- Failure rate is less than 0. 1 percent.
- The server-side latency is less than 1 second for at least 75 percent of the requests.
- Database server CPU utilization is less than 80 percent, which allows for 10 percent to be reserved for the Search crawl load, limited by using SQL Server Resource Governor.
- AxBxC (Graph notation): This is the number of Web servers, application servers, and database servers in a farm. So for example, 8x1x2 means that this environment has eight Web servers, one application server, and two database servers.
- VSTS Load: Threads used internally by Visual Studio Team System (VSTS) to simulate virtual users. We used increasing VSTS Load to generate more and more RPS for the topology.
Overview
Scaling approach
This section describes the specific order that we recommend for scaling computers in your environment, and it is the same approach we took for scaling this lab environment. This approach will allow you to find the best configuration for your workload and can be described as follows:
- First, we scaled out the Web servers. These were scaled out as far as possible under the tested workload, until the database server became the bottleneck and was not able to accommodate any more requests from the Web servers.
- Until this point, content database and services databases (user profile database, Social database, etc.) were all on the same database server. When we noticed that the database server was the bottleneck, we scaled out the database server by moving the content databases to another database server. At this point, the Web servers were not creating sufficient load on the database servers, so they were scaled out further.
- In lab environment, we did not test scale out further. But, if you need more scale, then the next logical step will be to have two computers share application server responsibilities.
We started off with a minimal farm configuration of one front-end Web server, one application server, and one SQL Server-based computer. Through multiple iterations, we finally ended at eight front-end servers, one application server, two SQL Server farm configurations. In the “Results and Analysis” section, you will find a comparison of Green Zone and Max Zone performance characteristics across different iterations. Details of how we found out Green Zone and Max Zone for each iteration is covered in “Appendix”.
Correlating lab environment with a production environment
The lab environment outlined in this document is a smaller scale model of a production environment at Microsoft, and although there are significant differences between the two environments, it can be useful to look at them side by side because they are both My Site and social computing environments where the patterns observed should be similar.
The lab environment contains a dataset that closely mimics the dataset from the production environment. The workload that is used for testing is largely similar to the workload seen in the production environment with few notable differences.
The most notable of the differences is that in the lab environment. We use fewer distinct users to perform the operations, and we perform operations on a smaller number of user profiles compared to the production environment. Also, the lab test runs happen over a shorter period of time.
All this has an effect on how many cache hits we have for the User Profile cache that is maintained on the application server. User Profile Service caches recently used user profiles on the application server. The default size of this cache is 256 MB, which approximately translates into 500,000 user profiles. Because the number of user profiles that was used in testing was limited to 1,500, and the duration of the tests were less than the recycle time of the cache, we almost always had cache hits. Thus, the throughput numbers presented in this document are on the higher side. You should definitely account for cache misses in your environment and hence, expect a lower throughput number.
For a detailed case study of a production My Sites and social computing portal at Microsoft, see SharePoint 2010 Technical Case Study: Social Environment.
Test notes
This document provides results from a test lab environment. Because this was a lab environment and not a production environment, we were able to control certain factors to show specific aspects of performance for this workload. In addition, certain elements of the production environment, in the following list, were left out of the lab environment to simplify testing overhead. Note that we do not recommend omitting these elements for production environments.
- Between test runs, we modified only one variable at a time, to make it easy to compare results between test runs.
- The database servers used in this lab environment were not part of a cluster because redundancy was not necessary for the purposes of these tests.
Search crawl was not running during the tests, whereas it might be running in a production environment. To take this into account, we lowered the SQL Server CPU utilization in our definition of Green Zone and Max to accommodate the resources that a search crawl would have consumed if it had been running simultaneously with our tests.
Test setup
Hardware
The following table presents hardware specifications for the computers that were used in this testing. Every front-end Web server (WFE) that was added to the server farm during multiple iterations of the test complies to the same specifications.
Front-end Web server |
Application server |
Database server |
|
|
Server model |
PE 2950 |
PE 2950 |
Dell PE 6850 |
|
Processor(s ) |
2px4c@2.33 GHz |
2px4c@2.33 GHz |
4px4c@ 3.19 GHz |
|
RAM |
8 GB |
8 GB |
32 GB |
|
Number of NICs |
2 |
2 |
1 |
|
NIC speed |
1 Gigabit |
1 Gigabit |
1 Gigabit |
|
Load balancer type |
F5 – Hardware load balancer |
n/a |
n/a |
|
ULS Logging level |
Medium |
Medium |
n/a |
Table 1: Hardware specifications for server computers
Software
The following table explains the software that was installed and running on the servers that were used in this testing.
Front-end Web server |
Application server |
Database server |
|
Operating System |
Windows Server® 2008 R2 x64 |
Windows Server 2008 R2 x64 |
Windows Server 2008 x64 |
Software version |
Microsoft SharePoint 4763.1000 (RTM), Office Web Applications 4763.1000 (RTM) |
Microsoft SharePoint 4763.1000 (RTM), WAC 4763.1000 (RTM) |
SQL Server 2008 R2 CTP3
|
Load balancer type |
F5 – Hardware load balancer |
n/a |
n/a |
ULS Logging level |
Medium |
Medium |
n/a |
Antivirus Settings |
Disabled |
Disabled |
Disabled |
Table 2: Software specifications for server computers
Topology and configuration
The following topology diagram explains the hardware setup that was used for the tests.
Diagram 1: Farm Configuration
Refer to Diagram 1 for the services that are provisioned in the test environment.
Dataset and disk geometry
The test farm was populated with a total of 166.5 GB of MySite content, evenly distributed across 10 content databases, 27.7 GB of Profile database content, 3.7 GB of Social database content (GUIDs for Social tags, notes and ratings) and 0.14 GB of Metadata Management database content (text for social tags and corresponding GUIDs).
The following table explains the dataset in detail:
|
Number of user profiles |
~150K |
|
Average number of memberships / user |
74 |
|
Average number of direct reports / user |
6 |
|
Average number of colleagues / user |
28 |
|
Number of total profile properties |
101 |
|
Number of multivalue properties |
21 |
|
Number of audiences |
130 |
|
Number of MySites |
~10K |
|
Number of blog sites |
~600 |
|
Total number of events in activity feed |
798K* |
|
Number of social tags/ratings |
5.04M** |
Table 3: Dataset detail
*
Social tagging study from del.icio.us suggests that an active user creates 4.2 tags/month. (Tags here mean any activity of assigning metadata to URLs, and hence includes keyword tags, ratings and notes.) This means an active user creates 4.2/30 = 0.14 tags/day. Assuming 1/3rd users of the social portal are actively tagging, we have 150K/3*0.14 tagging events per day. Activity feed tables maintain activity for 14 days, hence total number of tagging activity in the activity feed table comes to 150K/3*0.14*14. In addition to tagging events, if we assume that active user generates 1 more event per day such as a profile property update or status update, we have 150K/3*1*14 events added to activity feed tables. Thus, total number of events in activity feed tables comes to 150K/3*1.14*14 = 798K Among that, 98K of events is tagging activities which may trigger security trimming; rest of the events will be randomly distributed among status update and profile property changes.
**
Assume 1/3 of population are active users, each create 4.2 tags / month, where a tag can mean a keyword tag, a note or a rating. Assuming the farm is in use for 2 years, total tags will be 150K/3 * 4.2 * 12 * 2 = 5.04M.
The table below explains the disk geometry in details:
|
Database |
ContentDB 1, 2, 3, 4 |
ContentDB 5, 6 |
ContentDB 7, 8 |
ContentDB 9, 10 |
Profile |
Social |
Metadata |
|
Database Size |
61.4 GB |
39 GB |
32.3 GB |
33.7 GB |
27.7 GB |
3.7G B |
0.14 |
|
RAID configuration |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
|
Number of spindles for MDF |
1 |
1 |
1 |
1 |
6 |
1 |
1 |
|
Number of spindles for LDF |
one physical spindle shared by all databases |
||||||
Table 4: Disk geometry detail
Transactional mix
Important notes
- The tests only model prime time usage on a typical social computing portal. We did not consider the cyclical changes in user generated traffic that is seen with day-night cycles. Timer jobs, which require significant resources, such as Profile Synchronization and People Search Crawl, were tested independently with the same test workload to identify their citizenship effect.
- This test focuses more on social operations, such as newsfeeds, social tagging, and reading people profiles. It does have a small amount of typical collaboration traffic, but that is not the focus. We expect these results to help in designing a separate portal dedicated to My Sites and social features.
- Test mix does not include traffic from Search Content Crawl. However this was factored into our tests by modifying the Green Zone definition to be 40 percent SQL Server CPU usage as opposed to the standard 50 percent to allow 10percent for the search crawl. Similarly, we used 80 percent SQL Server CPU as the criteria for max RPS.
- In addition to the test mix listed in the following table, we also added eight RPS per front-end Web server for Outlook Social Connector traffic. We had security trimming turned ON, and we saw Secure Token Service being stressed as we approached about 8 RPS of Outlook Social Connector’s traffic on single front-end Web server to get activities of colleagues. This is a function of the dataset, test workload, and hardware we used in lab for testing, and you might see entirely different behavior. To avoid further stress on Secure Token Service, we decided to add Outlook Social Connector traffic as a function of the number of front-end Web servers in each iteration. Thus for 1X1X1, we have eight RPS of Outlook Social Connector traffic, while for 2X1X1 we have 16 RPS of Outlook Social Connector traffic, and so on.
Overall transaction mix is presented in the following table:
|
Description |
Read/write |
% of mix |
|
Add a colleague |
Write |
2.11% |
|
Create a rating on a URL, write a note or tag a URL |
Write |
3.22% |
|
List operations document |
Read/Write |
2.36% |
|
Get published links to model client calls to PublishedLinksService.asmx |
Read |
6.92% |
|
Get RSS feeds from lists |
Read |
3.72% |
|
View all items in document libraries and lists on My Site |
Read |
1.07% |
|
View a blog post |
Read |
0.04% |
|
View various My Site pages (my content, colleagues, newsfeed, my profile, someone else’s profile, organization browser, memberships, tags, and notes) |
Read |
3.87% |
|
Sync for Shared OneNote files |
Read |
10.0% |
|
Edit my profile page or status message, update picture |
Write |
2.31% |
|
Office Web Applications: Open and scroll files (PowerPoint®, Word, Excel®) |
Read |
0.13% |
|
List sync with Outlook® |
Read |
48.16% |
|
Upload a document |
Write |
0.09% |
|
Loading pages, document libraries, folders from Content DB |
Read |
15.93% |
|
Co-authoring of documents |
Read/Write |
0.17% |
Table 5: Transactional Mix
Additional Outlook Social Connector scenario test mix generating 8 RPS per front-end Web server:
|
Auto-sync my colleagues |
Read |
4% |
|
Auto-sync my colleagues’ news feeds |
Read |
96% |
Table 6: Outlook Social Connector scenario text mix
Results and analysis
Comparison of all iterations
As mentioned earlier, we started off with a minimal farm configuration of one front-end Web server, one application server and one SQL Server-based computer. Through multiple iterations, we finally ended at eight front-end Web servers, one application server, two SQL Server-based farm configurations. For each of these iterations, we performed step load tests to identify Green Zone and Max Zone. Details of step load tests within each iteration are provided in the Appendix. In the following table, you will find comparison of these Green Zone and Max Zone performance characteristics across different iterations.
The following table and charts provide a summary for comparison and analysis.
Green Zone results:
First let’s take a look at Green Zone performance characteristics across topologies. The following table provides a summary of results:
|
Topology |
1x1x1 |
2x1x1 |
3x1x1 |
5x1x1 |
8x1x1 |
8x1x2 |
|
Green Zone RPS |
137.25 |
278.08 |
440.72 |
683.07 |
793.67 |
873.4 |
|
Green Zone 75th Percentile Latency |
0.12 |
0.16 |
0.14 |
0.16 |
0.31 |
0.32 |
|
Green Zone front-end Web server CPU |
47.84 |
46.88 |
48.68 |
46.13 |
31.79 |
36.90 |
|
Green Zone application server CPU |
9.45 |
18.88 |
26.91 |
35.58 |
48.73 |
47.20 |
|
Green Zone SQL Server CPU |
5.45 |
10.61 |
16.46 |
24.73 |
30.03 |
32.40 (17.9 for Content DB and 14.5 for Services DB) |
Table 7:Green Zone performance
The following chart presents variation in CPU utilizations plotted on RPS, and offered by different topologies for Green Zone results.
From the chart above:
- RPS increased throughout as we added more computers to topologies.
- It is clear that front-end Web server CPU was the driving factor leading the topology to the boundary of Green Zone until 5X1X1, and at 8X1X1 application server CPU reached the boundary for Green Zone before the front-end Web servers could reach Green Zone boundaries.
- Throughout, SQL Server CPU was in a very healthy territory.
Max Zone results:
The following table provides a summary of results across topologies for Max Zone.
|
1x1x1 |
2x1x1 |
3x1x1 |
5x1x1 |
8x1x1 |
8x1x2 |
|
|
Max Zone RPS |
203.28 |
450.75 |
615.00 |
971.13 |
1655 |
1877 |
|
Max Zone Latency |
0.22 |
0.23 |
0.22 |
0.22 |
0.31 |
0.32 |
|
Max Zone front-end Web server CPU |
75.13 |
78.17 |
70.00 |
67.02 |
67 |
71.6 |
|
Max Zone application server CPU |
12.97 |
27.07 |
28.40 |
48.28 |
67.1 |
73.4 |
|
Max Zone SQL Server CPU |
7.64 |
16.06 |
21.00 |
38.38 |
79.5 |
74.9 (45.9 for Content DB and 29 for Services DB) |
Table 8: Results across topologies for Max Zone
The following chart presents variation in CPU utilizations plotted on RPS, and offered by different topologies for Max Zone results.
From the preceding chart:
- RPS increased throughout as we added more computers to topologies.
- It is clear that front-end Web server CPU was the bottleneck until 5X1X1, and at 8X1X1 SQL CPU became the bottleneck.
- Initially, application server CPU utilization was higher than SQL Server CPU utilization, but it is apparent that the growth rate of SQL Server CPU utilization is more than the growth rate of application server CPU utilization. At higher throughput, SQL Server CPU utilization overtook application server CPU utilization and became the bottleneck.
Green Zone vs. Max Zone:
The following charts compare throughput and latencies for Green Zone and Max Zone across different topologies.
From the charts above:
- Latencies don’t vary much with throughput or topologies. In our testing we saw all latencies under 0.5 seconds, which is very acceptable.
- Throughput increase is almost linear.
A note on I/Ops:
The following table and chart presents I/Ops observed on each database in different topologies. We did not run into disk I/O as a bottleneck, and looking at the trend, we did not record the data for later topologies.
|
|
1x1x1 Max Zone |
2x1x1 Max Zone |
3x1x1 Max Zone |
5x1x1 Max Zone |
|
Reads/Sec (ContentDB) |
21.33 |
20.80 |
24.24 |
22.42 |
|
Reads/Sec (ProfileDB) |
14.97 |
17.20 |
19.82 |
13.50 |
|
Reads/Sec (SocialDB) |
1.81 |
1.83 |
2.10 |
2.01 |
|
Writes/Sec (ContentDB) |
50.12 |
76.24 |
80.02 |
99.16 |
|
Writes/Sec (ProfileDB) |
9.01 |
24.31 |
23.35 |
38.29 |
|
Writes/Sec (SocialDB) |
4.12 |
9.47 |
10.63 |
19.45 |
Table 9: Observed I/Ops
Effect of people search crawl
We wanted to measure the effect of people search crawl on throughput offered by a configuration and by end user latencies. For this testing, we used results given by 8X1X1 configuration as baseline and started the incremental people search crawl. The incremental crawl indexed 49,375 items in 53 minutes.
Comparison of performance characteristics exhibited by the 8X1X1 configuration with and without people search incremental crawl are presented in the following table:
|
Baseline 8X1X1 Green Zone results |
8X1X1 with People Search crawl Green Zone results |
|
|
Throughput [RPS] |
1024.00 |
1026.00 |
|
Front-end Web server CPU [%] |
39.84 |
41.6 |
|
Application server CPU [%] |
41.40 |
43.1 |
|
Content/Service SQL Server CPU [%] |
36.63 |
39.5 |
|
Indexer CPU [%] |
0.52 |
34.6 |
|
Search SQL CPU [%] |
3.62 |
14.8 |
Table 10: Comparison of performance characteristics
From the table above:
- RPS nearly remained the same. Because there was no resource bottleneck in the 8X1X1 Green Zone, there is no reason for RPS to be affected.
- The front-end Web server and Content/Service SQL Server CPU utilization became only slightly higher.
-
Search Indexer and SQL Server CPU increased from 0.5% to 34.6%, and 3.6% to 14.8%.
Analysis
Application Server Scale
You might have noticed that in none of the configurations did we find application server as a bottleneck. Further, if you see application server CPU utilization for different VSTS loads in any single configuration, you will notice that it grows and then flattens out. An ideal example of this is seen in the 8X1X1 configuration (detailed results are in Appendix):
|
VSTS Load |
416 |
616 |
816 |
1016 |
1216 |
1416 |
1616 |
|
Application server CPU |
37.6 |
49.4 |
57.9 |
61.9 |
67.1 |
65.3 |
63.10 |
This is expected. In the case of a social portal, most of the operations require dealing with a SharePoint service called User Profile Service. Most of the operations require fetching a user’s profile from Profile DB that is provisioned when User Profile Service is created.
To avoid frequent SQL Server round trips, application server for User Profile Service maintains a cache of User Profiles. Initially, as the test environment is warming up, this cache is empty, and the application server is responding to incoming requests from the front-end Web server by constantly fetching User Profiles from SQL Server. These profiles are then cached, and subsequently, all requests from the front-end Web server can be responded to by the application server without causing a SQL Server round trip, by just looking up in the cache.
Because the number of user profiles used in testing was limited, we saw the application server warm up to cache all those user profiles, hence it showed an increasing utilization. When all the profiles were cached, it was a steady operation of cache lookups, and hence we see the application server CPU utilization stabling down.
Outlook Social Connector traffic and security trimming
Outlook Social Connector is an add-in that ships with Office 2010, which shows activities by your SharePoint Colleagues in Outlook. This add-in is also available for free download for Office 2007 and Office 2003.
Outlook Social Connector pings SharePoint server once every hour to fetch activities by colleagues of the user who is using it. It caches those activities for the hour. Next hour, it only asks for the delta of activities since the last time it called SharePoint. Thus, it follows a very predictable traffic pattern. For a 100,000-people deployment of Outlook Social Connector and SharePoint, assuming everyone is using it all day long, it generates 100,000 requests per hour, which translates to 27.77 requests per second.
Showing activities by other people leads to a possibility of information disclosure; if the URL that is tagged by a colleague is something confidential that a user does not have access to, then the user can find out about existence of that confidential piece of content by seeing it in Outlook Social Connector. To prevent this information disclosure, SharePoint filters all activities and shows only those URLs in activities that a user has access to. This filtering is what we call security trimming. It is ON by default, but it can be turned off.
Not every activity requires security trimming. Out of 16 activity type SharePoint supports, only 4 (tagging, note board comments, rating and DL membership changes) require security trimming. Also, because Outlook Social Connector asks only for a delta of activities that have happened since last time it synced, the number of activities per user that would require security trimming would be reasonably low.
Every request from Outlook Social Connector requiring security trimming results in an authenticated WCF call to Search Service’s application server. To get the authentication token for making this call, a WCF call is initially made to Secure Token Service.
We found out that if the Outlook Social Connector RPS goes beyond eight RPS per front-end Web server, Secure Token Service was under stress. This might or might not happen to each customer, because it is affected by the number of total users and total social tagging being made to a user’s colleagues. In the dataset we created, and the users we used, we probably had enough activities requiring security trimming that we saw this happen. Hence, we increased Outlook Social Connector traffic as a function of the number of front-end Web servers available. For the 1X1X1 configuration, we generated 8 RPS of Outlook Social Connector traffic, while for a 2X1X1 configuration we generated 16 RPS of Outlook Social Connector traffic, and so on.
This means, for the dataset, test mix, and hardware we had for testing, we could support about 8*60*60, that is, 28,800 requests per hour. With the way Outlook Social Connector works, this means that we could have supported 28,800 employees using Outlook Social Connector on a single front-end Web server with security trimming ON. Similarly, we could support 28,800*3, which is 86,400 employees using Outlook Social Connector on three front-end Web servers with security trimming ON.
This should help you estimate the hardware that is required to support Outlook Social Connector traffic, but keep in mind that the results we saw are specific to the dataset, test mix, and hardware we used for testing. Also, keep in mind that you have the option of turning off security trimming using PowerShell, or changing the frequency of Outlook Social Connector sync with SharePoint. Both of these options will have significant effect on hardware requirements.
Recommendations
Overall, in our testing, we found that:
- The environment scaled up to eight front-end Web servers for one application server and one database server; increase in throughput was almost linear throughout. After eight front-end Web servers, there are no additional gains to be made in throughput by adding more front-end Web servers because the bottleneck at this point was the database server CPU utilization.
- Further scale can be achieved by separating content database and services database on two separate database servers.
- We maxed out the 8x1x2 topology. At that point both the front end Web server and the application server CPU utilization was bottleneck. That leads us to believe that for the given hardware, dataset, and test workload, max RPS possible is represented by Max Zone RPS for 8x1x2, which is about 1877.
- Looking at the trends, it seems possible to extract the same throughput with a healthy farm, if the bottlenecks on front-end Web server and application server are addressed. The front end Web server bottleneck can be addressed by adding more front-end Web servers, and the application server bottleneck can be addressed by using two computers to play the role of application server. We did not try it out in the lab though.
- Latency is not affected by throughput or hardware variations.
- If you have security trimming turned ON, one front-end Web server can support about 8-10 RPS of Outlook Social Connector traffic. This means, one front-end Web server can support about 28,000 to 36,000 employees using Outlook Social Connector all day. Thus, if you are rolling out Outlook Social Connector to 100,000 employees, you can support the traffic generated by three front-end Web servers. These values can vary depending on social tagging usage at your company. If you imagine your company to have less social tagging activity than what we used in the dataset for this testing effort, you can get better throughput per front-end Web server.
- Incremental people search crawl doesn’t have much effect on the farm’s throughput as long as farm is maintained in a healthy state.
Appendix
Results from iterations
1 X 1 X 1 topology
Summary of results
- In addition to the test mix presented above, this farm had eight RPS traffic of Outlook Social Connector asking for feed events by a user.
- On a one front-end Web servers, one application server and one SQL Server-based farm, clearly the front-end Web server was the bottleneck. As presented in the data in the following table, front-end Web server CPU reached about 90 percent utilization when the farm was subjected to RPS of 238, using the transactional mix that is described earlier in this document.
- This configuration delivered Green Zone RPS of 137.25, with 75th percentile latency being 0.12 sec, and front-end Web servers CPU hovering around 47.8 percent utilization. This indicates that this farm can healthily deliver an RPS of about 137.25. Max Zone RPS delivered by this farm was 203.2 with latencies of 0.22 sec and front-end Web server CPU hovering around 85 percent.
- Because the front-end Web server was bottlenecked, for the next iteration, we added another front-end Web server to the farm.
Performance counters and graphs
Various performance counters captured during testing a 1 X 1 X 1 farm, at different steps in VSTS load, are presented below.
|
VSTS Load |
52 |
77 |
102 |
127 |
152 |
177 |
|
RPS |
99.8 |
147 |
188 |
218 |
238 |
243 |
|
Front-end Web server CPU |
33.9 |
50 |
71.8 |
81.1 |
90.8 |
89 |
|
Application Server CPU |
7.92 |
11.7 |
13.5 |
14.1 |
13.9 |
13.3 |
|
SQL Server CPU |
4.7 |
6.48 |
7.99 |
8.21 |
8.41 |
8.88 |
|
75th Percentile [sec] |
0.13 |
0.16 |
0.17 |
0.25 |
0.3 |
0.45 |
|
95th Percentile [sec] |
0.29 |
0.47 |
0.41 |
0.55 |
0.55 |
0.77 |
Table 1: Performance counters in a 1X1X1 farm configuration
2 X 1 X 1 farm configuration
Summary of results
- In addition to the test mix presented above, this farm had 16 RPS traffic of Outlook Social Connector asking for feed events by a user.
- On a two front-end Web server, one application server, and one SQL Server-based farm, the front-end Web server was the bottleneck. As presented in the data below, the front-end Web server CPU reached about 89 percent utilization when the farm was subjected to RPS of 520, using the transactional mix described earlier in this document.
- This configuration delivered Green Zone RPS of 278, with 75th percentile latency being 0.16 sec, and the front-end Web server CPU hovering around 47 percent utilization. This indicates that this farm can healthily deliver an RPS of about 278 with the test mix and hardware used for tests. Max Zone RPS delivered by this farm was 450 with latencies of 0.23 sec and the front-end Web server CPU hovering around 78 percent.
- Because the front-end Web server CPU was the bottleneck in this iteration, we relived the bottleneck by adding another front-end Web server for the next iteration.
Performance counters and graphs
Various performance counters captured during testing 2 X 1 X 1 farm, at different steps in VSTS load, are presented in the following table and chart.
|
VSTS Load |
104 |
154 |
204 |
254 |
304 |
354 |
|
RPS |
190 |
278 |
390 |
455 |
500 |
520 |
|
Front-end Web server CPU |
36 |
50.9 |
71.9 |
86.9 |
87.1 |
89.5 |
|
Application server CPU |
16 |
24.9 |
28.3 |
26.5 |
26.5 |
24.9 |
|
SQL Server CPU |
8.06 |
10.6 |
14.2 |
16.4 |
17.9 |
18.9 |
|
75th Percentile [sec] |
0.16 |
0.22 |
0.22 |
0.33 |
0.42 |
0.53 |
|
95th Percentile [sec] |
0.42 |
0.64 |
0.51 |
0.69 |
0.73 |
0.89 |
Table 2: Performance Counters during 2 X 1 X 1 configuration
3 X 1 X 1 farm configuration
Summary of results
- In addition to the test mix presented above, this farm had 24 RPS traffic of Outlook Social Connector asking for feed events by a user.
- On a three front-end Web server, one application server, and one SQL Server-based farm, the front-end Web server was the bottleneck. As presented in the data below, the front-end Web server CPU reached about 76 percent utilization when the farm was subjected to RPS of 629, using the transactional mix described earlier in this document.
- This configuration delivered Green Zone RPS of 440, with 75th percentile latency being 0.14 sec, and the front-end Web server CPU hovering around 48 percent utilization. This indicates that this farm can healthily deliver an RPS of about 440 with the test mix and hardware used for tests. Max Zone RPS delivered by this farm was 615 with latencies of 0.22 sec and the front-end Web server CPU hovering around 70 percent.
-
Because the front-end Web server CPU was the bottleneck in this iteration, we decided to add more front-end Web servers. Considering the delta between iterations seen previously by addition of a front-end Web server, we decided to add two front-end Web servers. We hoped to find application server or SQL Server as a bottleneck by doing so.
Performance counters and graphs
Various performance counters captured during testing the 3 X 1 X 1 farm, at different steps in VSTS load, are presented in the following table and charts.
|
VSTS Load |
156 |
231 |
306 |
381 |
456 |
531 |
|
RPS |
264 |
393 |
532 |
624 |
634 |
629 |
|
Front-end Web server CPU |
30.5 |
46.3 |
62.55 |
72.95 |
75.4 |
76 |
|
Application server CPU |
22.7 |
35.6 |
34.2 |
32.5 |
32.5 |
29.4 |
|
SQL Server CPU |
10.4 |
14.8 |
20.8 |
22.5 |
22.8 |
22.4 |
|
75th Percentile [sec] |
0.17 |
0.26 |
0.27 |
0.28 |
0.31 |
0.40 |
|
95th Percentile [sec] |
0.63 |
1.08 |
0.76 |
0.68 |
0.88 |
0.98 |
Table 3: Performance counters during 3X1X1 configuration
5 X 1 X 1 farm configuration
Summary of results
- In addition to the test mix presented above, this farm had 40 RPS traffic of Outlook Social Connector asking for feed events by a user.
- On a five front-end Web server, one application server, and one SQL Server-based farm, we saw significant increase in SQL Server CPU and application server CPU utilization, but still, the front-end Web server CPU was the bottleneck. As presented in the data below, the front-end Web server CPU reached about 88 percent utilization when the farm was subjected to RPS of 1315, using the transactional mix described earlier in this document.
- This configuration delivered Green Zone RPS of 683, with 75th percentile latency being 0.16 sec, and the front-end Web server CPU hovering around 46 percent utilization. This indicates that this farm can healthily deliver an RPS of about 683 with the test mix and hardware used for tests. Max Zone RPS delivered by this farm was 971 with latencies of 0.22 sec and the front-end Web server CPU hovering around 68percent.
- Looking at the trend, we decided to add three front-end Web servers and test for 8X1X1 configuration. We hoped to find application server or SQL Server as a bottleneck with that configuration
Performance counters and graphs
Various performance counters captured during testing 5 X 1 X 1 farm, at different steps in user load, are presented below. Because we saw no significant effect of VSTS load or configuration changes on latency, we stopped recording it.
|
VSTS Load |
260 |
385 |
510 |
635 |
760 |
885 |
|
RPS |
359 |
560 |
901 |
1188 |
1281 |
1315 |
|
front-end Web server CPU |
20.5 |
34 |
56.2 |
77.5 |
86.1 |
88 |
|
Application server CPU |
40.2 |
50.6 |
66.9 |
71.3 |
66.3 |
58.7 |
|
SQL Server CPU |
13.9 |
20.3 |
34.9 |
53.6 |
58.4 |
64 |
Table 4: Performance counters during 5X1X1 configuration
8 X 1 X 1 farm configuration
Summary of results
- In addition to the test mix presented above, this farm had 64 RPS traffic of Outlook Social Connector asking for feed events by a user.
- On eight front-end Web servers, one application server, and one SQL Server-based farm, finally, SQL Server CPU was the bottleneck. As presented in the data below, SQL Server CPU reached about 80 percent utilization when the farm was subjected to RPS of 1664, using the transactional mix described earlier in this document.
- This configuration delivered Green Zone RPS of 793, with 75th percentile latency being 0.31 sec, and SQL Server CPU hovering around 30 percent utilization, while application server CPU was about 48 percent. This indicates that this farm can healthily deliver an RPS of about 793 with the test mix and hardware used for tests. Max Zone RPS delivered by this farm was 1655 with latencies of 0.31 sec and SQL Server CPU hovering around 80 percent.
- Because SQL Server CPU was the bottleneck in this iteration, we relived the bottleneck by separating out the content database and services database on two different instances of SQL Server for the next iteration.
Performance counters and graphs
Various performance counters captured during testing the 8 X 1 X 1 farm, at different steps in VSTS load, are presented in the following table and chart.
|
VSTS Load |
416 |
616 |
816 |
1016 |
1216 |
1416 |
1616 |
|
RPS |
664 |
1101 |
1359 |
1530 |
1655 |
1664 |
1617.00 |
|
Front-end Web server CPU |
26.7 |
44.4 |
54.7 |
61.5 |
67 |
65.9 |
65.10 |
|
Application server CPU |
37.6 |
49.4 |
57.9 |
61.9 |
67.1 |
65.3 |
63.10 |
|
SQL Server CPU |
23.2 |
42 |
57.9 |
69.5 |
79.5 |
80.8 |
77.30 |
Table 5: Performance counters during 8X1X1 configuration
8 X 1 X 2 farm configuration
Summary of results
- In addition to the test mix presented above, this farm had 64 RPS traffic of Outlook Social Connector asking for feed events by a user.
- On an eight front-end Web server, one application server, and two SQL Server-based farms, we could take the configuration to its extreme. The front-end Web server and application server, both were bottlenecked, while combined SQL Server utilization was also in the higher 70s. The farm exhibited RPS of 1817 at max.
- This was the last iteration we tried. But clearly, if you need more scale, the next step would be to use two computers to perform application server duties. That would allow you to have many more front-end Web servers, and hence load on each front-end Web server will be less.
Performance counters and graphs
Various performance counters captured during testing 8 X 1 X 2 farm, at different steps in VSTS load, are presented in the following table and chart.
|
VSTS Load |
466 |
666 |
866 |
1066 |
1266 |
1416 |
|
RPS |
466.00 |
873.40 |
1431.00 |
1703.00 |
1766.00 |
1817.00 |
|
Front-end Web server CPU |
19.90 |
36.90 |
57.60 |
68.00 |
71.40 |
71.60 |
|
Application server CPU |
29.80 |
47.20 |
63.50 |
71.40 |
71.90 |
73.40 |
|
Total SQL Server CPU |
19.61 |
32.40 |
55.20 |
63.60 |
68.50 |
74.90 |
|
Content SQL Server CPU |
9.93 |
17.90 |
31.90 |
40.10 |
42.30 |
45.90 |
|
Services SQL Server CPU |
9.68 |
14.50 |
23.30 |
23.50 |
26.20 |
29.00 |
Table 6: Performance counters during 8X1X2 configuration
5 Tips for a Smooth SSIS Upgrade to SQL Server 2012
5 Tips for a Smooth SSIS Upgrade to SQL Server 2012
SQL Server Technical Article
Writers: Runying Mao, Carla Sabotta
Technical Reviewers: David Noor, Jeanne Baker
Published: November 2011
Applies to: SQL Server 2012
Summary:
Microsoft SQL Server 2012 Integration Services (SSIS) provides significant improvements in both the developer and administration experience. This article provides tips that can help to make the upgrade to Microsoft SQL Server 2012 Integration Services successful. The tips address editing package configurations and specifically connection strings, converting configurations to parameters, converting packages to the project deployment model, updating Execute Package tasks to use project references and parameterizing the PackageName property.
Copyright
This document is provided “as-is”. Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it.
Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.
© 2011 Microsoft. All rights reserved.
Contents
1TIP #1: Edit Package Configuration and Data Source after upgrading 4
TIP #2: Convert to project deployment model using Project Conversion Wizard 6
TIP #5: Convert package configuration to parameter when possible 10
Introduction
Microsoft SQL Server 2012 Integration Services (SSIS) provides significant improvements in both the developer and administration experience. New SSIS features have been introduced in order to improve developer productivity, and simplify the deployment, configuration, management and troubleshooting of SSIS packages.
SQL Server 2012 Integration Services introduces the project as a self-contained, deployment unit. Common values can be shared among packages in the same project through project parameters and project connection managers. ETL developers can easily reference child packages that are inside the project.
Solutions that were created in earlier versions of SSIS (pre-SQL Server 2012) will be supported in SQL Server 2012. When you upgrade the solutions, you can take advantage of the new SQL Server 2012 features. Although in SQL Server 2012 SSIS offers wizards for upgrading most solution components, there will be a few settings that you’ll need to change manually.
Here are a few tips that can help to make the upgrade successful.
TIP #1: Edit Package Configuration and Data Source after upgrading
The first step to upgrade an SSIS solution is to run the SSIS Package Upgrade Wizard. The SSIS Package Upgrade Wizard makes appropriate changes to package properties and upgrades the package format.
The wizard launches when you open a pre-SQL Server 2012 package in the SQL Server Data Tools for the first time. SQL Server Data Tools replaces (BIDs). The wizard can also be launched manually by running SSISUpgrade.exe, which is located under %ProgramFiles%\Microsoft SQL Server\110\DTS\Binn.
It is critical to note that the SSIS Package Upgrade Wizard does not upgrade settings such as connection strings that are defined in the package configurations. After a package upgrade, you may need to make some manual changes to the package configuration to run the upgraded package successfully.
For example, you have an SSIS 2005 package. The package uses an OLE DB connection manager to connect to the AdventureWorks database in a local SQL Server 2005 instance. The package also uses an XML package configuration file to dynamically configure the ConnectionString property of the OLE DB connection manager. The following shows the contents of the XML package configuration file.
You have set up a machine with a standalone SQL Server 2012 installation. You move the SSIS 2005 package to the machine and run the SSIS Package Upgrade Wizard to upgrade the package to SQL Server 2012. When the wizard finishes, you need to manually change the provider name from SQLNCLI.1 to SQLNCLI11.1 in the XML package configuration file to run the upgraded package successfully. The wizard does not update package configuration files.
If you don’t update the provider name in the configuration file, the file configures the OLE DB connection manager to use the SQLNCLI.1 provider that is the SQL Server 2005 Native Client Library. SQLNCLI11.1 is the SQL Server 2012 Native Client Library. Because the SQL Server 2005 Native Client Library is not included in SQL Server 2012, the following error message will appear when you open or execute the upgraded package on the machine where SQL Server 2012 is installed:
The requested OLE DB provider SQLNCLI.1 is not registered. If the 32-bit driver is not installed, run the package in 64-bit mode. Error code: 0x00000000. An OLE DB record is available. Source: “Microsoft OLE DB Service Components” Hresult: 0x80040154 Description: “Class not registered”.
So, if your pre-SQL Server 2012 package uses any kind of package configurations, it is important to remember that you may need to manually update the content of the package configurations after you upgrade the package to SQL Server 2012. This applies to the different types of configurations, such as XML configuration files.
Connection strings that require updates and are stored in data source files or set by expressions, need to be updated manually.
TIP #2: Convert to project deployment model using Project Conversion Wizard
SQL Server 2012 SSIS supports two deployment models: the package deployment model and the project deployment model. The package deployment model was available in previous releases of SSIS and is the default deployment model for upgraded packages. In this model, the unit of deployment is the package. The project deployment model is new in SQL Server 2012 and provides additional package deployment and management features such as parameters and the Integration Services catalog. The unit of deployment is the project.
Please read Project Deployment Overview in SQL Server “Denali” CTP1 – SSIS (http://social.technet.microsoft.com/wiki/contents/articles/project-deployment-overview-in-sql-server-quot-denali-quot-ctp1-ssis.aspx ) for a detailed walk through as well as comparison between these two deployment models.
Read Projects in SQL Server “Denali” CTP1 – SSIS (http://social.technet.microsoft.com/wiki/contents/articles/projects-in-sql-server-denali-ctp1-ssis.aspx) for a thorough explanation of the new project concept.
To convert a package to the project deployment, right click the project in Solution Explorer and then click Convert to Project Deployment Model. The Project Conversion Wizard launches and walks you through the conversion process.
TIP #3: Update Execute Package Task to use project reference and use parameter to pass data from parent package to child package
If an SSIS package contains an Execute Package Task, the Project Conversion Wizard prompts you to update the task to use the project reference.
For example, your SSIS project contains several packages. Inside the project, one package (typically called the parent package) runs another package (typically called the child package) by using an Execute Package Task. In Pre-SQL Server 2012 releases of SSIS, the parent package references the child package by using a File connection manager. At deployment, you need to remember to update the File connection manager to ensure that it points to the new location of the child package.
In SQL Server 2012 Integration Services you can configure the parent package to reference the child package by name when the child package is included in the same project as the parent package. Using this project reference makes the deployment experience much smoother. You don’t need to remember to update the reference between the parent package and the child package at deployment. For a thorough explanation of the project reference in the Execute Package Task, please see Changes to the Execute Package Task (http://blogs.msdn.com/b/mattm/archive/2011/07/18/changes-to-the-execute-package-task.aspx).
In previous releases of SSIS, you pass data from the parent package to the child package by creating a package configuration that uses the parent variable configuration type. This enables a child package that is run from a parent package to access a variable in the parent.
It is recommended that you configure the Execute Package Task to use parameter binding to pass data from the parent package to the child package. Parameters make this task easier. For example, you want a parent package to dynamically determine the number of days in a current month and have the child package perform a task for that number of times. You can create a variable in the parent package that represents the number of days and create a parameter in the child package. Then in the Execute Package Task, you bind the parameter in the child package to the variable in the parent package.
Please read Parameters in SQL Server “Denali” CTP1 – SSIS (http://social.technet.microsoft.com/wiki/contents/articles/parameters-in-sql-server-denali-ctp1-ssis.aspx) for a description of parameters and the numerous benefits they offer.
TIP #4: Parameterize PackageName property of Execute Package Task to dynamically configure which child package to run at execution time
Suppose your SSIS 2008 package has an Execute Package Task, and the package uses a File connection manager to connect to a child package. You dynamically assign which child package the Execute Package Task runs by configuring the connection string property of the File connection manager.
The following is the content of the XML package configuration file used by your SSIS 2008 package.
When the Project Conversion Wizard converts the package to the project deployment model and updates the Execute Package Task to use the project reference, the File connection manager that was used to connect to the child package is no longer used by the Execute Package Task. To continue to dynamically determine which child package the task runs, you create a parameter and map that parameter to the PackageName property of the Execute Package Task as shown in the following image.
TIP #5: Convert package configuration to parameter when possible
Parameters are new to SQL Server 2012 Integration Services and are the replacement for package configurations. You use parameters to assign values to package properties, whether at design time or run time. The values are pushed to a package when it is executed rather than having the package pull values from the package configurations.
The Project Conversion Wizard prompts you to optionally convert package configurations to parameters. It is possible that you might choose to keep a package configuration as an intermediate step of upgrading to SQL Server 2012. When your package has both configuration values and parameter values, it is important to understand the order in which these values are applied. Package configuration values will be applied first. If there are also parameter values for the same properties, these values will be applied next and will overwrite the package configuration values.
Conclusion
Microsoft SQL Server 2012 Integration Services (SSIS) offers features that greatly enhance the development and administrative experience. These tips could help users ensure successful upgrades of their current solutions to SQL Server 2012 so that they can take advantage of SQL Server 2012’s new features. For more information about SQL Server 2012 Integration Services and what’s new, please refer to What’s New (Integration Services) (http://msdn.microsoft.com/en-us/library/bb522534(v=SQL.110).aspx).
Did this paper help you? Please give us your feedback. Tell us on a scale of 1 (poor) to 5 (excellent), how would you rate this paper and why have you given it this rating? For example:
- Are you rating it high due to having good examples, excellent screen shots, clear writing, or another reason?
- Are you rating it low due to poor examples, fuzzy screen shots, or unclear writing?
This feedback will help us improve the quality of white papers we release.
SharePoint 2010 Best Practices
Table of Contents
- Intro
- Performance
- Planning
- Installation, Removal, Configuration, and Operation
- Deployment
- Virtualization
- Real Life Usage
- Backup and Recovery
- Development
- Search
- Upgrade and Migration
- Extranet Environments
- Farms
- Top 10 Blogs to Follow
- Top 5 SharePoint Books
- Top 10 SharePoint Tools
- Top Forums/Communities
Intro
Best practices are, and rightfully so, always a much sought-after topic. There are various kinds of best practices:
- Microsoft best practices. In real life, these are the most important ones to know, as most companies implementing SharePoint best practices have a tendency to follow as much of these as possibly can. Independent consultants doing architecture and code reviews will certainly take a look at these as well. In general, you can safely say that best practices endorsed by Microsoft have an added bonus and it will be mentioned whenever this is the case.
- Best practices. These practices are patterns that have proven themselves over and over again as a way to achieve a high quality of your solutions, and it’s completely irrelevant who proposed them. Often MS best practices will also fall in this category. In real life, these practices should be the most important ones to follow.
- Practices. These are just approaches that are reused over and over again, but not necessarily the best ones. Wiki’s are a great way to discern best practices from practices. It’s certainly possible that this page refers to these “Practices of the 3rd kind”, but hopefully, the SharePoint community will eventually filter them out. Therefore, everybody is invited and encouraged to actively participate in the various best practices discussions.
This Wiki page contains an overview of SharePoint 2010 Best Practices of all kinds, divided by categories.
Performance
When implementing IT solutions, everybody will face the day where a customer isn’t happy with the way an application is performing. Because of the complex infrastructure and vast amount of features of SharePoint, there are many ways to approach these issues. Because of that and the importance of the topic, our first category outlines best practices to tackling performance problems.
- http://social.technet.microsoft.com/wiki/contents/articles/7926.sharepoint-2010-tips-for-dealing-with-performance-issues-en-us.aspx, perhaps the most comprehensive resource about solving SharePoint 2010 performance issues.
- http://www.dell.com/downloads/global/services/SharePoint_2010_Virtual_Small_Farm_Performance_Study_Final.pdf , performance study of virtualized small farm.
- http://www.f5.com/pdf/white-papers/dell-sharepoint-2010-large-farm-performance-study-wp.pdf , large farm performance study.
- http://www.microsoft.com/en-us/download/details.aspx?id=7480 , performance studies.
- http://www.cleverworkarounds.com/2012/05/31/demystifying-sharepoint-performance-management-part-1-how-to-stop-developers-blaming-the-infrastructure , great series of articles about performance management.
- http://www.microsoft.com/downloads/details.aspx?FamilyID=37B1333C-A8D9-45F9-BE78-9212C2CD3641&displaylang=e&displaylang=en Managing large, up to multi-terabyte content databases.
Planning
Every SharePoint undertaking will at one point face the following questions: how long will it take, how much will it cost to implement, and how will we use it?
- http://social.technet.microsoft.com/wiki/contents/articles/10590.sharepoint-2010-best-practices-estimating-projects.aspx, the first and so far only resource covering this particular topic.
- http://gallery.technet.microsoft.com/Maxer-for-SharePoint-2010-8cd0f26f , Maxer for SharePoint 2010 is a tool that checks for capacity boundaries in existing SharePoint farms.
- http://technet.microsoft.com/en-us/library/hh403882.aspx , capacity management.
- http://social.technet.microsoft.com/wiki/contents/articles/3495.typical-sharepoint-project-part-1-en-us.aspx, interpretation of a “typical” SharePoint project.
- http://social.technet.microsoft.com/wiki/contents/articles/8874.sharepoint-server-2010-getting-started-beginners-to-expert-en-us.aspx, getting started.
- http://www.microsoft.com/en-us/download/details.aspx?id=1288 , adoption best practices.
Installation, Removal, Configuration, and Operation
This section deals with best practices regarding the following questions: How to install SharePoint? How to configure it? How to keep it operating? All best practices are targeted towards the IT Pro.
- http://technet.microsoft.com/en-us/library/cc850692.aspx , provides advice on how to get and keep your environment up and running. This list is created by members of the Microsoft Consulting Services team.
- http://technet.microsoft.com/en-us/library/cc850694.aspx , provides advice for the IT Pro how to keep team collaboration sites manageable. Created by Microsoft Consulting Services.
- http://technet.microsoft.com/en-us/library/cc262706.aspx , discusses how to keep My Sites manageable. Created by Microsoft employees.
- http://technet.microsoft.com/en-us/library/hh292621.aspx , discusses content deployment best practices. Created by Microsoft employees.
- http://technet.microsoft.com/en-us/library/hh292622.aspx , SQL Server 2008 best practices. Created by Microsoft employees.
- http://technet.microsoft.com/en-us/library/hh377942.aspx , People and Profiles. Created by MS employees.
- http://technet.microsoft.com/en-us/library/cc850696.aspx , Search. Created by MS employees.
- http://seclists.org/basics/2008/Sep/225 , general considerations regarding service accounts.
- http://social.technet.microsoft.com/wiki/contents/articles/8217.sharepoint-2010-crawl-database-is-too-large.aspx, discusses what to do when the crawl database becomes too large.
- http://social.technet.microsoft.com/wiki/contents/articles/7999.sharepoint-2010-strategies-for-handling-historical-data-in-custom-lists.aspx, how to handle historical data in lists?
- http://social.technet.microsoft.com/wiki/contents/articles/10550.sharepoint-2010-best-practices-ad-groups-or-sharepoint-groups.aspx, should you choose to use AD groups or SharePoint groups?
- http://social.technet.microsoft.com/wiki/contents/articles/8778.sharepoint-2010-best-practices-is-sharepoint-designer-really-pure-evil.aspx, musings about the advantages of SPD.
- http://social.technet.microsoft.com/wiki/contents/articles/8723.sharepoint-2010-best-practices-list-throttling.aspx, be careful with list throttling settings.
- http://social.technet.microsoft.com/wiki/contents/articles/9628.sharepoint-2010-best-practice-row-limits-in-external-lists.aspx, row limits in external lists.
- http://social.technet.microsoft.com/wiki/contents/articles/9950.sharepoint-2010-best-practice-troubleshooting-an-ssrs-report-that-times-out-en-us.aspx, troubleshooting SSRS reports.
- http://www.gokanozcifci.be/subsite/how-to-install-sharepoint-2010-with-sql-2012-on-windows-server-8.html , installing SharePoint 2010 and SQL Server 2012.
- http://www.gokanozcifci.be/subsite/install-and-configure-sharepoint-13-preview.html Install and configure SharePoint 2013 and SQL server 2012
- http://social.technet.microsoft.com/wiki/contents/articles/11177.sharepoint-2010-best-practices-removing-sharepoint-from-servers.aspx, explains how to remove SharePoint in a clean and orderly fashion.
- http://technet.microsoft.com/en-us/library/ee662513.aspx , least privileges for administrative and service accounts.
- http://sharepointgeorge.com/2010/installing-sharepoint-2010-privilege-service-accounts , least privilege model for service accounts.
- http://en.hugon.ws/articles/sharepoint-2010/using-the-service-accounts-with-least-privilege-to-run-sharepoint-2010 , least privilege user accounts.
- http://technet.microsoft.com/en-us/library/cc678863.aspx , account permissions and security settings.
- http://www.sharepointblog.co.uk/2012/06/quickly-create-sharepoint-service-account/ , a PowerShell script for quickly creating SharePoint service accounts.
Deployment
Deployment of software artifacts is important. This section discusses best practices.
- http://technet.microsoft.com/en-us/library/cc303424.aspx , deployment to different farm topologies.
- http://blogs.technet.com/b/stefan_gossner/archive/2009/01/16/content-deployment-best-practices.aspx , Content deployment best practices
Virtualization
It’s very common that SharePoint farms use virtualization techniques. This section is dedicated to best practices concerning virtualization.
- http://technet.microsoft.com/en-us/library/hh295699.aspx , discusses virtualization best practices. Created by Microsoft Consulting Services.
- http://www.andrewconnell.com/blog/articles/SqeezePerformanceOutOfVirtualPCs.aspx , performance tips for virtual PCs.
Real Life Usage
Once you have SharePoint deployed, it’s up to the end users, power users, and IT Pros to make the best of it. This section discusses best practices targeted towards this audience.
- http://social.technet.microsoft.com/wiki/contents/articles/8110.sharepoint-2010-best-practices-asset-vs-picture-library.aspx, discussion of the use of Asset vs Picture libraries.
- http://social.technet.microsoft.com/wiki/contents/articles/8138.sharepoint-2010-best-practices-copying-permissions-between-site-collections.aspx, dealing with copying permissions between site collections.
- http://social.technet.microsoft.com/wiki/contents/articles/2209.best-practices-for-indexing-file-share-content-en-us.aspx, best practices that help indexing content stored on file shares.
- http://sharepointdragons.com/2012/03/27/sharepoint-2010-wcf-best-practice/ , discussion whether a WCF service sould be separate or run within SharePoint context.
- http://social.technet.microsoft.com/wiki/contents/articles/8680.sharepoint-2010-best-practices-folders-not-necessarily-considered-evil.aspx, discusses advantages of folders over views.
- http://social.technet.microsoft.com/wiki/contents/articles/9638.sharepoint-2010-best-practice-keep-relational-data-in-a-database-or-migrate-it-into-a-custom-list.aspx, discusses whether to store data in SharePoint lists, external lists, or relational databases.
Backup and Recovery
This section deals with best practices about the back up and restore of SharePoint environments.
- http://technet.microsoft.com/en-us/library/gg266384.aspx , backup and recovery.
- http://social.technet.microsoft.com/wiki/contents/articles/5099.sharepoint-2010-backup-and-recovery-guidelines-en-us.aspx, general backup and recovery guidelines.
- http://social.technet.microsoft.com/wiki/contents/articles/9726.sharepoint-2010-best-practice-when-should-you-back-up-sharepoint-via-sharepoint-central-administration-sca.aspx, discusses when to use SCA to perform backups.
- http://technet.microsoft.com/en-us/library/cc261687(office.14).aspx , provides good explanation of what get’s backed up and when.
Development
This section covers best practices targeted towards software developers.
- http://msdn.microsoft.com/en-us/library/ee554869(office.14).aspx , setting up a development environment.
- http://msdn.microsoft.com/en-us/library/ff770300.aspx , patterns and practices from MS Patterns and Practices group for developing applications in SharePoint 2010.
- http://msdn.microsoft.com/en-us/library/ee231582.aspx , Visual Studio.NET 2010 requirements for developing SharePoint 2010 solutions.
- http://msdn.microsoft.com/en-us/library/ee556427(v=office.14).aspx , best practices for SPF concerning suspending impersonation, unnecessary construction of SPSite and SPWeb objects, event receivers, disposing objects, file naming restrictions, handling large folders and lists, object caching techniques, and optimizing code performance.
- http://sharepointdragons.com/ , disposing objects.
- http://msdn.microsoft.com/en-us/library/ee558807(v=office.14).aspx , tips about writing efficient code.
- http://msdn.microsoft.com/en-us/library/gg552614.aspx , Security Best Practices for Developers in SharePoint 2010.
- http://msdn.microsoft.com/en-us/library/gg615455.aspx , Best Practices for Developing Sandboxed Solutions in SharePoint 2010
- http://spg.codeplex.com/ , SharePoint Guidance Hands on Labs by MS.
- http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=23919 developing applications for SharePoint 2010.
- http://charliedigital.com/2009/08/29/design-patterns-for-sharepoint-entry-1/ , creating an IDisposable class for lists so you won’t forget to dispose SPSite and SPWeb objects.
- http://sharepointmvc.codeplex.com/ , SharePoint MVC pattern.
- http://social.technet.microsoft.com/wiki/contents/articles/7884.sharepoint-2010-building-a-web-part-that-implements-the-mvp-pattern.aspx, SharePoint MVP pattern.
- http://social.technet.microsoft.com/wiki/contents/articles/8181.sharepoint-2010-best-practice-which-api-to-use.aspx, which API should you use: SharePoint object model/web services/client object model/custom WCF service leveraging server object model?
- http://sharepointdragons.com/2012/03/09/determine-the-current-user-in-a-sps-2010-workflow/ , best practice for determining the current user in a SharePoint workflow.
- http://social.technet.microsoft.com/wiki/contents/articles/8665.sharepoint-2010-best-practices-where-to-publish-a-wcf-service-en-us.aspx, create a separate WCF service or a WCF service in SharePoint context?
- http://social.technet.microsoft.com/wiki/contents/articles/8044.sps-2010-best-practices-retrieving-the-current-user-in-a-workflow.aspx, how to retrieve the current user in a SharePoint workflow.
- https://caf.sharepoint.microsoftonline.com/ MICROSOFT SHAREPOINT ONLINE CODE ANALYSIS FRAMEWORK (MSOCAF) This application have code rules that can be used to validate SharePoint Solutions.
- http://archive.msdn.microsoft.com/SPDisposeCheck SPDisposeCheck is a tool that helps developers and administrators check custom SharePoint solutions that use the SharePoint Object Model helping measure against known Microsoft dispose best practices.
Search
Search is a complex topic and important to almost every company working with SharePoint. This section discusses best practices.
- http://technet.microsoft.com/en-us/library/cc262794(v=office.12).aspx , when to crawl content.
- http://technet.microsoft.com/en-us/library/ff599528.aspx , FAST search topology.
- http://social.technet.microsoft.com/wiki/contents/articles/411.best-practices-for-using-a-dedicated-front-end-web-server-for-crawling-in-microsoft-office-sharepoint-server-2007-en-us.aspx , how to use a dedicated web server for crawling.
Upgrade and Migration
If a product is successful, it has to be upgraded at some point.
- http://technet.microsoft.com/en-us/library/cc261992.aspx , reviews upgrade best practices.
- http://www.slideshare.net/joeloleson/sharepoint-2010-upgrade-best-practices-croatia , video of best practices.
- http://technet.microsoft.com/en-us/video/initial-best-practices-lessons-learned-from-microsoft-its-sharepoint-2010-upgrade.aspx , video about Upgrade lessons learned by MS.
- http://www.aiim.org/community/blogs/expert/SharePoint-2010-Upgrade-Options-Considerations-Best-Practices , preparing for upgrade.
- http://channel9.msdn.com/posts/Joel-Oleson-SharePoint-2010-Upgrade–Migration-Best-Practices-SharepointPTDay-29102010 , video about upgrade and migration best practices.
Extranet Environments
This section provides an overview of planning and design considerations for SharePoint Extranet Environments.
- http://technet.microsoft.com/en-us/library/hh204611.aspx , provides advice regarding SharePoint Extranet environments. Created by members of Microsoft Consulting Services.
- http://www.microsoft.com/en-us/download/details.aspx?id=24079 , extranet topologies.
- http://blogs.kma-llc.net/microknowledge/2011/06/sharepoint-2010-extranet-topologies-and-firewalls.html , extranet topologies and firewalls.
- http://technet.microsoft.com/en-us/library/cc263513.aspx , more extranet topologies.
Farms
This section discusses best practices regarding SharePoint 2010 farm topologies.
- http://programming4.us/enterprise/3492.aspx , discussion of three supported topologies: Single Data Center High-Availability Model, Cross-Site High-Availability Model, and Multiple-Farm Cross-Site Model.
- http://technet.microsoft.com/en-us/library/cc263199.aspx , technical diagrams of recommended solutions.
- http://www.microsoft.com/en-us/download/details.aspx?id=6096 , common ways to build and scale farm topologies.
- http://www.techbubbles.com/sharepoint-2010/sharepoint-2010-farm-topologies/ , nice explanation of basic terminology.
- http://sharepointsolutions.com/sharepoint-help/blog/2011/02/how-to-scale-out-a-sharepoint-2010-farm-from-two-tier-to-three-tier-by-adding-a-dedicated-application-server/ , scale out from a 2-tier farm to a 3-tier farm.
- http://www.sharepointedutech.com/2011/01/02/topology-med-sp2010-farm/ , topology and licensing example of medium farm.
- http://www.astaticstate.com/2010/07/sharepoint-2010-physical-topology.html , examples of diagramming SharePoint topologies.
- http://technet.microsoft.com/en-us/library/ee805948.aspx , 3-tier farm.
- http://mscerts.programming4.us/sharepoint/Topologies%20for%20SharePoint%202010.aspx , discussion of basic types of deployment.
Top 10 Blogs to Follow
It’s certainly a best practice to keep up to date with the latest SharePoint news. Therefore, a top 10 of blog suggestions to follow is included.
- http://blogs.msdn.com/b/sharepointdev/ , the SharePoint Developer team blog.
- http://www.andrewconnell.com/blog , Andrew Connell on SharePoint.
- http://www.sharepointjoel.com/default.aspx , Joel Oleson’s SharePoint Land.
- http://sharepointdragons.com , Nikander & Margriet on SharePoint.
- http://blogs.msdn.com/b/uksharepoint/ , the SharePoint guys.
- https://www.nothingbutsharepoint.com/Pages/default.aspx , Nothing but SharePoint.
- http://www.shillier.com/default.aspx , Scot Hillier on SharePoint.
- http://www.lightningtools.com/blog/default.aspx , Lightning Tools Blog.
- http://www.wictorwilen.se/ , Wictor Wilen on SharePoint.
- http://www.gokanozcifci.be/blog , Gokan Ozcifci on SharePoint
Top 5 SharePoint Books
Books remain the most important resource for learning a new topic. Here’s a suggestion of the best SharePoint 2010 books out there.
- http://www.amazon.com/Inside-Microsoft-SharePoint-2010-Pattison/dp/0735627460/ref=sr_1_1?s=books&ie=UTF8&qid=1337663232&sr=1-1 , the favorite developer book about SharePoint 2010.
- http://www.amazon.com/Microsoft-SharePoint-2010-Administrators-Companion/dp/0735627207/ref=sr_1_2?s=books&ie=UTF8&qid=1337603828&sr=1-2 , great resource for administrators.
- http://www.amazon.com/SharePoint-2010-Site-Owners-Manual/dp/1933988754/ref=sr_1_1?s=books&ie=UTF8&qid=1337663093&sr=1-1 , does a great job teaching SharePoint end users and power users.
- http://www.amazon.com/Microsoft-SharePoint-Designer-2010-Step/dp/0735627339/ref=sr_1_2?s=books&ie=UTF8&qid=1337663155&sr=1-2 , dedicated to SharePoint Designer 2010.
- http://www.amazon.com/SharePoint-2010-Web-Parts-Action/dp/1935182773/ref=sr_1_1?s=books&ie=UTF8&qid=1337663192&sr=1-1 , best book out there about SharePoint 2010 web parts.
Top 10 SharePoint Tools
What to put in your bag of tools?
- http://cksdev.codeplex.com/ , CKSDev makes SharePoint development easier.
- http://spm.codeplex.com/ , SharePoint Manager is a SharePoint object model explorer.
- http://camldotnet.codeplex.com/ , CAML.NET: IntelliSense for CAML.
- http://karinebosch.wordpress.com/2012/05/12/caml-designer/ , CAML Designer makes creating CAML queries a lot easier (successor of U2U CAML Query Builder).
- http://archive.msdn.microsoft.com/ULSViewer , you’ll always need a tool to view the ULS log files. ULS Log Viewer is probably the most popular one of the lot.
- http://gallery.technet.microsoft.com/Maxer-for-SharePoint-2010-8cd0f26f , the SharePoint Maxer tool helps checking for capacity planning limits.
- http://gallery.technet.microsoft.com/The-SharePoint-Flavored-5b03f323 , the SharePoint Flavored Weblog Reader (SFWR) helps troubleshooting performance problems by analyzing the IIS log files of SharePoint WFEs.
- http://gallery.technet.microsoft.com/The-Migration-Dragon-for-628acae0 , the Migration Dragon for SharePoint 2010 is a tool that can help to migrate file and folder structures from the file system to SharePoint 2010 Document Libraries leveraging the batching mechanism of the SharePoint managed client object model.
- http://visualstudiogallery.msdn.microsoft.com/36a6eb45-a7b1-47c3-9e85-09f0aef6e879 , Muse.VSExtensions, a great tool for referencing assemblies located in the GAC.
- http://spservices.codeplex.com/ , jQuery Library for SharePoint Web Services
Microsoft SQL Server 2005 Analysis Services Performance Guide
Microsoft SQL Server 2005 Analysis Services Performance Guide
SQL Server Technical Article
Author: Elizabeth Vitt
Subject Matter Experts:
T.K. Anand
Sasha (Alexander) Berger
Marius Dumitru
Eric Jacobsen
Edward Melomed
Akshai Mirchandani
Mosha Pasumansky
Cristian Petculescu
Carl Rabeler
Wayne Robertson
Richard Tkachuk
Dave Wickert
Len Wyatt
Published: February 2007
Applies To: SQL Server 2005, Service Pack 2
Summary: This white paper describes how application developers can apply performance-tuning techniques to their Microsoft SQL Server 2005 Analysis Services Online Analytical Processing (OLAP) solutions.
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.
This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred.
2007 Microsoft Corporation. All rights reserved.
Microsoft, Windows, and Windows Server
are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.
All other trademarks are property of their respective owners.
Table of Contents
Understanding the querying architecture 8
Data retrieval: measure group data 15
Optimizing the dimension design 18
Identifying attribute relationships 18
Using hierarchies effectively 22
Maximizing the value of aggregations 24
How the Storage Engine uses aggregations 25
Why not create every possible aggregation? 27
How to interpret aggregations 29
Which aggregations are built 30
How to impact aggregation design 31
Suggesting aggregation candidates 32
Specifying statistics about cube data 36
Adopting an aggregation design strategy 39
Using partitions to enhance query performance 40
How partitions are used in querying 41
Aggregation considerations for multiple partitions 43
Specifying the calculation space 44
Taking advantage of the Query Execution Engine cache 58
Applying calculation best practices 60
Tuning Processing Performance 61
Understanding the processing architecture 61
Dimension-processing commands 64
Partition-processing commands 65
Refreshing dimensions efficiently 67
Optimizing the source query 67
Reducing attribute overhead 68
Optimizing dimension inserts, updates, and deletes 70
Refreshing partitions efficiently 71
Optimizing the source query 71
Using partitions to enhance processing performance 72
Optimizing data inserts, updates, and deletes 72
Evaluating rigid vs. flexible aggregations 73
Optimizing Special Design Scenarios 76
Special aggregate functions 76
Optimizing semiadditive measures 78
Complex dimension relationships 79
Near real-time data refreshes 86
Understanding how Analysis Services uses memory 92
Shrinkable vs. non-shrinkable memory 94
Memory demands during querying 95
Memory demands during processing 96
Increasing available memory 97
Monitoring memory management 97
Minimizing metadata overhead 98
Monitoring the timeout of idle sessions 99
Tuning memory for partition processing 100
Understanding how Analysis Services uses CPU resources 103
Processor demands during querying 104
Processor demands during processing 104
Maximize parallelism during querying 105
Maximize parallelism during processing 107
Use a load-balancing cluster 109
Understanding how Analysis Services uses disk resources 110
Disk resource demands during processing 110
Disk resource demands during querying 110
Disabling unnecessary logging 111
Appendix A – For More Information 113
Appendix B – Partition Storage Modes 113
Multidimensional OLAP (MOLAP) 113
Appendix C – Aggregation Utility 116
Benefits of the Aggregation Utility 116
How the Aggregation Utility organizes partitions 117
How the Aggregation Utility works 118
Introduction
Fast query response times and timely data refresh are two well-established performance requirements of Online Analytical Processing (OLAP) systems. To provide fast analysis, OLAP systems traditionally use hierarchies to efficiently organize and summarize data. While these hierarchies provide structure and efficiency to analysis, they tend to restrict the analytic freedom of end users who want to freely analyze and organize data on the fly.
To support a broad range of structured and flexible analysis options, Microsoft® SQL Server™ Analysis Services (SSAS) 2005 combines the benefits of traditional hierarchical analysis with the flexibility of a new generation of attribute hierarchies. Attribute hierarchies allow users to freely organize data at query time, rather than being limited to the predefined navigation paths of the OLAP architect. To support this flexibility, the Analysis Services OLAP architecture is specifically designed to accommodate both attribute and hierarchical analysis while maintaining the fast query performance of conventional OLAP databases.
Realizing the performance benefits of this combined analysis paradigm requires understanding how the OLAP architecture supports both attribute hierarchies and traditional hierarchies, how you can effectively use the architecture to satisfy your analysis requirements, and how you can maximize the architecture’s utilization of system resources.
Note To apply the performance tuning techniques discussed in this white paper, you must have SQL Server 2005 Service Pack 2 installed.
To satisfy the performance needs of various OLAP designs and server environments, this white paper provides extensive guidance on how you can take advantage of the wide range of opportunities to optimize Analysis Services performance. Since Analysis Services performance tuning is a fairly broad subject, this white paper organizes performance tuning techniques into the following four segments.
Enhancing Query Performance – Query performance directly impacts the quality of the end user experience. As such, it is the primary benchmark used to evaluate the success of an OLAP implementation. Analysis Services provides a variety of mechanisms to accelerate query performance, including aggregations, caching, and indexed data retrieval. In addition, you can improve query performance by optimizing the design of your dimension attributes, cubes, and MDX queries.
Tuning Processing Performance – Processing is the operation that refreshes data in an Analysis Services database. The faster the processing performance, the sooner users can access refreshed data. Analysis Services provides a variety of mechanisms that you can use to influence processing performance, including efficient dimension design, effective aggregations, partitions, and an economical processing strategy (for example, incremental vs. full refresh vs. proactive caching).
Optimizing Special Design Scenarios
– Complex design scenarios require a distinct set of performance tuning techniques to ensure that they are applied successfully, especially if you combine a complex design with large data volumes. Examples of complex design components include special aggregate functions, parent-child hierarchies, complex dimension relationships, and “near real-time” data refreshes.
Tuning Server Resources
–
Analysis Services operates within the constraints of available server resources. Understanding how Analysis Services uses memory, CPU, and disk resources can help you make effective server management decisions that optimize querying and processing performance.
Three appendices provide links to additional resources, information on various partition storage modes, and guidance on using the Aggregation Utility that is a part of SQL Server 2005 Service Pack 2 samples.
Enhancing Query Performance
Querying is the operation where Analysis Services provides data to client applications according to the calculation and data requirements of a MultiDimensional eXpressions (MDX) query. Since query performance directly impacts the user experience, this section describes the most significant opportunities to improve query performance. Following is an overview of the query performance topics that are addressed in this section:
Understanding the querying architecture – The
Analysis Services querying architecture supports three major operations: session management, MDX query execution, and data retrieval. Optimizing query performance involves understanding how these three operations work together to satisfy query requests.
Optimizing the dimension design – A well-tuned dimension design is perhaps one of the most critical success factors of a high-performing Analysis Services solution. Creating attribute relationships and exposing attributes in hierarchies are design choices that influence effective aggregation design, optimized MDX calculation resolution, and efficient dimension data storage and retrieval from disk.
Maximizing the value of aggregations – Aggregations improve query performance by providing precalculated summaries of data. To maximize the value of aggregations, ensure that you have an effective aggregation design that satisfies the needs of your specific workload.
Using partitions to enhance query performance – Partitions provide a mechanism to separate measure group data into physical units that improve query performance, improve processing performance, and facilitate data management. Partitions are naturally queried in parallel; however, there are some design choices and server property optimizations that you can specify to optimize partition operations for your server configuration.
Writing efficient MDX – This section describes techniques for writing efficient MDX statements such as: 1) writing statements that address a narrowly defined calculation space, 2) designing calculations for the greatest re-usage across multiple users, and 3) writing calculations in a straight-forward manner to help the Query Execution Engine select the most efficient execution path.
Understanding the querying architecture
To make the querying experience as fast as possible for end users, the Analysis Services querying architecture provides several components that work together to efficiently retrieve and evaluate data. Figure 1 identifies the three major operations that occur during querying: session management, MDX query execution, and data retrieval as well as the server components that participate in each operation.
Figure 1 Analysis Services querying architecture
Session management
Client applications communicate with Analysis Services using XML for Analysis (XMLA) over TCP IP or HTTP. Analysis Services provides an XMLA listener component that handles all XMLA communications between Analysis Services and its clients. The Analysis Services Session Manager controls how clients connect to an Analysis Services instance. Users authenticated by Microsoft® Windows and who have rights to Analysis Services can connect to Analysis Services. After a user connects to Analysis Services, the Security Manager determines user permissions based on the combination of Analysis Services roles that apply to the user. Depending on the client application architecture and the security privileges of the connection, the client creates a session when the application starts, and then reuses the session for all of the user’s requests. The session provides the context under which client queries are executed by the Query Execution Engine. A session exists until it is either closed by the client application, or until the server needs to expire it. For more information regarding the longevity of sessions, see Monitoring the timeout of idle sessions in this white paper.
MDX query execution
The primary operation of the Query Execution Engine is to execute MDX queries. This section provides an overview of how the Query Execution Engine executes queries. To learn more details about optimizing MDX, see Writing efficient MDX later in this white paper.
While the actual query execution process is performed in several stages, from a performance perspective, the Query Execution engine must consider two basic requirements: retrieving data and producing the result set.
- Retrieving data—To retrieve the data requested by a query, the Query Execution Engine decomposes each MDX query into data requests. To communicate with the Storage Engine, the Query Execution Engine must translate the data requests into subcube requests that the Storage Engine can understand. A subcube represents a logical unit of querying, caching, and data retrieval. An MDX query may be resolved into one or more subcube requests depending on query granularity and calculation complexity. Note that the word subcube is a generic term. For example, the subcubes that the Query Execution Engine creates during query evaluation are not to be confused with the subcubes that you can create using the MDX CREATE SUBCUBE statement.
-
Producing the result set—To manipulate the data retrieved from the Storage Engine, the Query Execution Engine uses two kinds of execution plans to calculate results: it can bulk calculate an entire subcube, or it can calculate individual cells. In general, the subcube evaluation path is more efficient; however, the Query Execution Engine ultimately selects execution plans based on the complexities of each MDX query. Note that a given query can have multiple execution plans for different parts of the query and/or different calculations involved in the same query. Moreover, different parts of a query may choose either one of these two types of execution plans independently, so there is not a single global decision for the entire query. For example, if a query requests resellers whose year-over-year profitability is greater than 10%, the Query Execution Engine may use one execution plan to calculate each reseller’s year-over-year profitability and another execution plan to only return those resellers whose profitability is greater than 10%.
When you execute an MDX calculation, the Query Execution Engine must often execute the calculation across more cells than you may realize. Consider the example where you have an MDX query that must return the calculated year-to-date sales across the top five regions. While it may seem like you are only returning five cell values, Analysis Services must execute the calculation across additional cells in order to determine the top five regions and also to return their year to date sales. A general MDX optimization technique is to write MDX queries in a way that minimizes the amount of data that the Query Execution Engine must evaluate. To learn more about this MDX optimization technique, see Specifying the calculation space later in this white paper.
As the Query Execution Engine evaluates cells, it uses the Query Execution Engine cache and the Storage Engine Cache to store calculation results. The primary benefits of the cache are to optimize the evaluation of calculations and to support the re-usage of calculation results across users. To optimize cache re-usage, the Query Execution Engine manages three cache scopes that determine the level of cache reusability: global scope, session scope, and query scope. For more information on cache sharing, see Taking advantage of the Query Execution Engine cache in this white paper.
Data retrieval: dimensions
During data retrieval, the Storage Engine must efficiently choose the best mechanism to fulfill the data requests for both dimension data and measure data.
To satisfy requests for dimension data, the Storage Engine extracts data from the dimension attribute and hierarchy stores. As it retrieves the necessary data, the Storage Engine uses dynamic on-demand caching of dimension data rather than keeping all dimension members statically mapped into memory. The Storage Engine simply brings members into memory as they are needed. Dimension data structures may reside on disk, in Analysis Services memory, or in the Windows operating system file cache, depending on memory load of the system.
As the name suggests, the Dimension Attribute Store contains all of the information about dimension attributes. The components of the Dimension Attribute Store are displayed in Figure 2.
Figure 2 Dimension Attribute Store
As displayed in the diagram, the Dimension Attribute Store contains the following components for each attribute in the dimension:
- Key store—The key store contains the attribute’s key member values as well as an internal unique identifier called a DataID. Analysis Services assigns the DataID to each attribute member and uses the same DataID to refer to that member across all of its stores.
- Property store—The property store contains a variety of attribute properties, including member names and translations. These properties map to specific DataIDs. DataIDs are contiguously allocated starting from zero. Property stores as well as relationship stores (relationship stores are discussed in more detail later) are physically ordered by DataID, in order to ensure fast random access to the contents, without additional index or hash table lookups.
- Hash tables—To facilitate attribute lookups during querying and processing, each attribute has two hash tables which are created during processing and persisted on disk. The Key Hash Table indexes members by their unique keys. A Name Hash Table indexes members by name.
- Relationship store—The Relationship store contains an attribute’s relationships to other attributes. More specifically, the Relationship store stores each source record with DataID references to other attributes. Consider the following example for a product dimension. Product is the key attribute of the dimension with direct attribute relationships to color and size. The data instance of product Sport Helmet, color of Black, and size of Large may be stored in the relationship store as 1001, 25, 5 where 1001 is the DataID for Sport Helmet, 25 is the Data ID for Black, and 5 is the Data ID for Large. Note that if an attribute has no attribute relationships to other attributes, a Relationship Store is not created for that particular attribute. For more information on attribute relationships, see Identifying attribute relationships in this whitepaper.
- Bitmap indexes—To efficiently locate attribute data in the Relationship Store at querying time, the Storage Engine creates bitmap indexes at processing time. For each DataID of a related attribute, the bitmap index states whether or not a page contains at least one record with that DataID. For attributes with a very large number of DataIDs, the bitmap indexes can take some time to process. In most scenarios, the bitmap indexes provide significant querying benefits; however, there is a design scenario where the querying benefit that the bitmap index provides does not outweigh the processing cost of creating the bitmap index in the first place. It is possible to remove the bitmap index creation for a given attribute by setting the AttributeHierarchyOptimizedState property to Not Optimized. For more information on this design scenario, see Reducing attribute overhead in this white paper.
In addition to the attribute store, the Hierarchy Store arranges attributes into navigation paths for end users as displayed in Figure 3.
Figure 3 Hierarchy stores
The Hierarchy store consists of the following primary components:
- Set Store—The Set Store uses DataIDs to construct the path of each member, mapped from the first level to the current level. For example, All, Bikes, Mountain Bikes, Mountain Bike 500 may be represented as 1,2,5,6 where 1 is the DataID for All, 2 is the DataID for Bikes, 5 is the DataID for Mountain Bikes, and 6 is the DataID for Mountain Bike 500.
- Structure Store—For each member in a level, the Structure Store contains the DataID of the parent member, the DataID of the first child, and the total children count. The entries in the Structure Store are ordered by each member’s Level index. The Level index of a member is the position of a member in the level as specified by the ordering settings of the dimension. To better understand the Structure Store, consider the following example. If Bikes contains 3 children, the entry for Bikes in the Structure Store would be 1,5,3 where 1 is the DataID for the Bike’s parent, All, 5 is the DataID for Mountain Bikes, and 3 is the number of Bike’s children.
Note that only natural hierarchies are materialized in the hierarchy store and optimized for data retrieval. Unnatural hierarchies are not materialized on disk. For more information on the best practices for designing hierarchies, see Using hierarchies effectively.
Data retrieval: measure group data
For data requests, the Storage Engine retrieves measure group data that is physically stored in partitions. A partition contains two categories of measure group data: fact data and aggregations. To accommodate a variety of data storage architectures, each partition can be assigned a different storage mode that specifies where fact data and aggregations are stored. From a performance perspective, the storage mode that provides the fastest query performance is the Multidimensional Online Analytical Processing (MOLAP) storage mode. In MOLAP, the partition fact data and aggregations are stored in a compressed multidimensional format that Analysis Services manages. For most implementations, MOLAP storage mode should be used; however, if you require additional information about other partition storage modes, see Appendix B. If you are considering a “near real-time” deployment, see Near real-time data refreshes in this white paper.
In MOLAP storage, the data structures for fact data and aggregation data are identical. Each is divided into segments. A segment contains a fixed number of records (typically 64 KB) divided into 256 pages with 256 records in each. Each record stores all of the measures in the partition’s measure group and a set of internal DataIDs that map to the granularity attributes of each dimension. Only records that are present in the relational fact table are stored in the partition, resulting in highly compressed data files.
To efficiently fulfill data requests, the Storage Engine follows an optimized process to satisfy the request by using three general mechanisms, Storage Engine Cache, aggregations, and fact data represented in Figure 4:.
Figure 4 Satisfying data requests
Figure 4 presents a data request for {(Europe, 2005), (Asia, 2005)}. To fulfill this request, the Storage Engine chooses among the following approaches:
- Storage Engine cache—The Storage Engine first attempts to satisfy the data request using the Storage Engine cache. Servicing a data request from the Storage Engine cache provides the best query performance. The Storage Engine cache always resides in memory. For more information on managing the Storage Engine cache, see Memory demands during querying.
- Aggregations—If relevant data is not in the cache, the Storage Engine checks for a precalculated data aggregation. In some scenarios, the aggregation may exactly fit the data request. For example, an exact fit occurs when the query asks for sales by category by year and there is an aggregation that summarizes sales by category by year. While an exact fit is ideal, the Storage Engine can use also data aggregated at a lower level, such as sales aggregated by month and category or sales aggregated by quarter and item. The Storage Engine then summarizes the values on the fly to produce sales by category by year. For more information on how to design aggregations to improve performance, see Maximizing the value of aggregations.
- Fact data—If appropriate aggregations do not exist for a given query, the Storage Engine must retrieve the fact data from the partition. The Storage Engine uses many internal optimizations to effectively retrieve data from disk including enhanced indexing and clustering of related records. For both aggregations and fact data, different portions of data may reside either on disk or in the Windows operating system file cache, depending on memory load of the system.
A key performance tuning technique for optimizing data retrieval is to reduce the amount of data that the Storage Engine needs to scan by using multiple partitions that physically divide your measure group data into distinct data slices. Using multiple partitions can not only enhance querying speed, but they can also provide greater scalability, facilitate data management, and optimize processing performance.
From a querying perspective, the Storage Engine can predetermine the data stored in each MOLAP partition and optimize which MOLAP partitions it scans in parallel. In the example in Figure 4, a partition with 2005 data is displayed in blue and a partition with 2006 data is displayed in orange. The data request displayed in the diagram {(Europe, 2005), (Asia, 2005)} only requires 2005 data. Consequently, the Storage Engine only needs to go to the 2005 partition. To maximize query performance, the Storage Engine uses parallelism, such as scanning partitions in parallel, wherever possible. To locate data in a partition, the Storage Engine queries segments in parallel and uses bitmap indexes to efficiently scan pages to find the desired data.
Partitions are a major component of high-performing cubes. For more information on the broader benefits of partitions, see the following sections in this white paper:
- For more information on how bitmap indexes are created and used in partitions, see Partition-processing jobs.
- To find out more detailed information about the querying benefits of partitioning, see Using partitions to enhance query performance.
- To learn more about the processing benefits of using partitions, see Using partitions to enhance processing performance.
Optimizing the dimension design
A well-tuned dimension design is one of the most critical success factors of a high-performing Analysis Services solution. The two most important techniques that you can use to optimize your dimension design for query performance are:
- Identifying attribute relationships
- Using hierarchies effectively
Identifying attribute relationships
A typical data source of an Analysis Services dimension is a relational data warehouse dimension table. In relational data warehouses, each dimension table typically contains a primary key, attributes, and, in some cases, foreign key relationships to other tables.
Table 1 Column properties of a simple Product dimension table
|
Dimension table column |
Column type |
Relationship to primary key |
Relationship to other columns |
|
Product Key |
Primary Key |
Primary Key |
— |
|
Product SKU |
Attribute |
1:1 |
— |
|
Description |
Attribute |
1:1 |
— |
|
Color |
Attribute |
Many:1 |
— |
|
Size |
Attribute |
Many:1 |
Many: 1 to Size Range |
|
Size Range |
Attribute |
Many:1 |
|
|
Subcategory |
Attribute |
Many:1 |
Many: 1 to Category |
|
Category |
Attribute |
Many:1 |
Table 1 displays the design of a simple product dimension table. In this simple example, the product dimension table has one primary key column, the product key. The other columns in the dimension table are attributes that provide descriptive context to the primary key such as product SKU, description, and color. From a relational perspective, all of these attributes either have a many-to-one relationship to the primary key or a one-to-one relationship to the primary key. Some of these attributes also have relationships to other attributes. For example, size has a many-to-one relationship with size range and subcategory has a many-to-one relationship with category.
Just as it is necessary to understand and define the functional dependencies among fields in relational databases, you must also follow the same practices in Analysis Services. Analysis Services must understand the relationships among your attributes in order to correctly aggregate data, effectively store and retrieve data, and create useful aggregations. To help you create these associations among your dimension attributes, Analysis Services provides a feature called attribute relationships. As the name suggests, an attribute relationship describes the relationship between two attributes.
When you initially create a dimension, Analysis Services auto-builds a dimension structure with many-to-one attribute relationships between the primary key attribute and every other dimension attribute as displayed in Figure 5.
Figure 5 Default attribute relationships
The arrows in Figure 5 represent the attribute relationships between product key and the other attributes in the dimension. While the dimension structure presented in Figure 5 provides a valid representation of the data from the product dimension table, from a performance perspective, it is not an optimized dimension structure since Analysis Services is not aware of the relationships among the attributes.
With this design, whenever you issue a query that includes an attribute from this dimension, data is always summarized from the primary key and then grouped by the attribute. So if you want sales summarized by Subcategory, individual product keys are grouped on the fly by Subcategory. If your query requires sales by Category, individual product keys are once again grouped on the fly by Category. This is somewhat inefficient since Category totals could be derived from Subcategory totals. In addition, with this design, Analysis Services doesn’t know which attribute combinations naturally exist in the dimension and must use the fact data to identify meaningful member combinations. For example, at query time, if a user requests data by Subcategory and Category, Analysis Services must do extra work to determine that the combination of Subcategory: Mountain Bikes and Category: Accessories does not exist.
To optimize this dimension design, you must understand how your attributes are related to each other and then take steps to let Analysis Services know what the relationships are.
To enhance the product dimension, the structure in Figure 6 presents an optimized design that more effectively represents the relationships in the dimension.
Figure 6 Product dimension with optimized attribute relationships
Note that the dimension design in Figure 6 is different than the design in Figure 5. In Figure 5, the primary key has attribute relationships to every other attribute in the dimension. In Figure 6, two new attribute relationships have been added between Size and Size Range and Subcategory and Category.
The new relationships between Size and Size Range and Subcategory and Category reflect the many-to-one relationships among the attributes in the dimension. Subcategory has a many-to-one relationship with Category. Size has a many-to-one relationship to Size Range. These new relationships tell Analysis Services how the nonprimary key attributes (Size and Size Range, and Subcategory and Category) are related to each other.
Typically many-to-one relationships follow data hierarchies such as the hierarchy of products, subcategories, and categories depicted in Figure 6. While a data hierarchy can commonly suggest many-to-one relationships, do not automatically assume that this is always the case. Whenever you add an attribute relationship between two attributes, it is important to first verify that the attribute data strictly adheres to a many-to-one relationship. As a general rule, you should create an attribute relationship from attribute A to attribute B if and only if the number of distinct (a, b) pairs from A and B is the same (or smaller) than the number of distinct members of A. If you create an attribute relationship and the data violates the many-to-one relationship, you will receive incorrect data results.
Consider the following example. You have a time dimension with a month attribute containing values such as January, February, March and a year attribute containing values such as 2004, 2005, and 2006. If you define an attribute relationship between the month and year attributes, when the dimension is processed, Analysis Services does not know how to distinguish among the months for each year. For example, when it comes across the January member, it does not which year should it roll it up to. The only way to ensure that data is correctly rolled up from month to year is to change the definition of the month attribute to month and year. You make this definition change by changing the KeyColumns property of the attribute to be a combination of month and year.
The KeyColumns property consists of a source column or combination of source columns (known as a collection) that uniquely identifies the members for a given attribute. Once you define attribute relationships among your attributes, the importance of the KeyColumns property is highlighted. For every attribute in your dimension, you must ensure that the KeyColumns property of each attribute uniquely identifies each attribute member. If the KeyColumns property does not uniquely identify each member, duplicates encountered during processing are ignored by default, resulting in incorrect data rollups.
Note that if the attribute relationship has a default Type of Flexible, Analysis Services does not provide any notification that it has encountered duplicate months and incorrectly assigns all of the months to the first year or last year depending on data refresh technique. For more information on the Type property and the impact of your data refresh technique on key duplicate handling, see Optimizing dimension inserts, updates, and deletes in this white paper.
Regardless of your data refresh technique, key duplicates typically result in incorrect data rollups and should be avoided by taking the time to set a unique KeyColumns property. Once you have correctly configured the KeyColumns property to uniquely define an attribute, it is a good practice to change the default error configuration for the dimension so that it no longer ignores duplicates. To do this, set the KeyDuplicate property from IgnoreError to ReportAndContinue or ReportAndStop. With this change, you can be alerted of any situation where the duplicates are detected.
Whenever you define a new attribute relationship, it is critical that you remove any redundant relationships for performance and data correctness. In Figure 6, with the new attribute relationships, the Product Key no longer requires direct relationships to Size Range or Category. As such, these two attribute relationships have been removed. To help you identify redundant attribute relationships, Business Intelligence Development Studio provides a visual warning to alert you about the redundancy; however, it does not require you to eliminate the redundancy. It is a best practice to always manually remove the redundant relationship. Once you remove the redundancy, the warning disappears.
Even though Product Key is no longer directly related to Size Range and Category, it is still indirectly related to these attributes through a chain of attribute relationships. More specifically, Product Key is related to Size Range using the chain of attribute relationships that link Product Key to Size and Size to Size Range. This chain of attribute relationships is also called cascading attribute relationships.
With cascading attribute relationships, Analysis Services can make better performance decisions concerning aggregation design, data storage, data retrieval, and MDX calculations. Beyond performance considerations, attribute relationships are also used to enforce dimension security and to join measure group data to nonprimary key granularity attributes. For example, if you have a measure group that contains sales data by Product Key and forecast data by Subcategory, the forecast measure group will only know how to roll up data from Subcategory to Category if attribute relationships exists between Subcategory and Category.
The core principle behind designing effective attribute relationships is to create the most efficient dimension model that best represents the semantics of your business. While this section provides guidelines and best practices for optimizing your dimension design, to be successful, you must be extremely familiar with your data and the business requirements that the data must support before considering how to tune your design.
Consider the following example. You have a time dimension with an attribute called Day of Week. This attribute contains seven members, one for each day of the week, where the Monday member represents all of the Mondays in your time dimension. Given what you learned from the month / year example, you may think that you should immediately change the KeyColumns property of this attribute to concatenate the day with the calendar date or some other attribute. However, before making this change, you should consider your business requirements. The day-of-week grouping can be valuable in some analysis scenarios such as analyzing retail sales patterns by the day of the week. However, in other applications, the day of the week may only be interesting if it is concatenated with the actual calendar date. In other words, the best design depends on your analysis scenario. So while it is important to follow best practices for modifying dimension properties and creating efficient attribute relationships, ultimately you must ensure that your own business requirements are satisfied. For additional thoughts on various dimension designs for a time dimension, see the blog Time calculations in UDM: Parallel Period.
Using hierarchies effectively
In Analysis Services, attributes can be exposed to users by using two types of hierarchies: attribute hierarchies and user hierarchies. Each of these hierarchies has a different impact on the query performance of your cube.
Attribute hierarchies are the default hierarchies that are created for each dimension attribute to support flexible analysis. For non parent-child hierarchies, each attribute hierarchy consists of two levels: the attribute itself and the All level. The All level is automatically exposed as the top level attribute of each attribute hierarchy.
Note that you can disable the All attribute for a particular attribute hierarchy by using the IsAggregatable property. Disabling the All attribute is generally not advised in most design scenarios. Without the All attribute, your queries must always slice on a specific value from the attribute hierarchy. While you can explicitly control the slice by using the Default Member property, realize that this slice applies across all queries regardless of whether your query specifically references the attribute hierarchy. With this in mind, it is never a good idea to disable the All attribute for multiple attribute hierarchies in the same dimension.
From a performance perspective, attributes that are only exposed in attribute hierarchies are not automatically considered for aggregation. This means that no aggregations include these attributes. Queries involving these attributes are satisfied by summarizing data from the primary key. Without the benefit of aggregations, query performance against these attributes hierarchies can be somewhat slow.
To enhance performance, it is possible to flag an attribute as an aggregation candidate by using the Aggregation Usage property. For more detailed information on this technique, see Suggesting aggregation candidates in this white paper. However, before you modify the Aggregation Usage property, you should consider whether you can take advantage of user hierarchies.
In user hierarchies, attributes are arranged into predefined multilevel navigation trees to facilitate end user analysis. Analysis Services enables you to build two types of user hierarchies: natural and unnatural hierarchies, each with different design and performance characteristics.
- In a natural hierarchy, all attributes participating as levels in the hierarchy have direct or indirect attribute relationships from the bottom of the hierarchy to the top of the hierarchy. In most scenarios, natural hierarchies follow the chain of many-to-one relationships that “naturally” exist in your data. In the product dimension example discussed earlier in Figure 6, you may decide to create a Product Grouping hierarchy that from bottom-to-top consists of Products, Product Subcategories, and Product Categories. In this scenario, from the bottom of the hierarchy to the top of the hierarchy, each attribute is directly related to the attribute in the next level of the hierarchy. In an alternative design scenario, you may have a natural hierarchy that from bottom to top consists of Products and Product Categories. Even though Product Subcategories has been removed, this is still a natural hierarchy since Products is indirectly related to Product Category via cascading attribute relationships.
- In an unnatural hierarchy the hierarchy consists of at least two consecutive levels that have no attribute relationships. Typically these hierarchies are used to create drill-down paths of commonly viewed attributes that do not follow any natural hierarchy. For example, users may want to view a hierarchy of Size Range and Category or vice versa.
From a performance perspective, natural hierarchies behave very differently than unnatural hierarchies. In natural hierarchies, the hierarchy tree is materialized on disk in hierarchy stores. In addition, all attributes participating in natural hierarchies are automatically considered to be aggregation candidates. This is a very important characteristic of natural hierarchies, important enough that you should consider creating natural hierarchies wherever possible. For more information on aggregation candidates, see Suggesting aggregation candidates.
Unnatural hierarchies are not materialized on disk and the attributes participating in unnatural hierarchies are not automatically considered as aggregation candidates. Rather, they simply provide users with easy-to-use drill-down paths for commonly viewed attributes that do not have natural relationships. By assembling these attributes into hierarchies, you can also use a variety of MDX navigation functions to easily perform calculations like percent of parent. An alternative to using unnatural hierarchies is to cross-join the data by using MDX at query time. The performance of the unnatural hierarchies vs. cross-joins at query time is relatively similar. Unnatural hierarchies simply provide the added benefit of reusability and central management.
To take advantage of natural hierarchies, you must make sure that you have correctly set up cascading attribute relationships for all attributes participating in the hierarchy. Since creating attribute relationships and creating hierarchies are two separate operations, it is not uncommon to inadvertently miss an attribute relationship at some point in the hierarchy. If a relationship is missing, Analysis Services classifies the hierarchy as an unnatural hierarchy, even if you intended it be a natural hierarchy.
To verify the type of hierarchy that you have created, Business Intelligence Development Studio issues a warning icon 
whenever you create a user hierarchy that is missing one or more attribute relationships. The purpose of the warning icon is to help identify situations where you have intended to create a natural hierarchy but have inadvertently missed attribute relationships. Once you create the appropriate attribute relationships for the hierarchy in question, the warning icon disappears. If you are intentionally creating an unnatural hierarchy, the hierarchy continues to display the warning icon to indicate the missing relationships. In this case, simply ignore the warning icon.
In addition, while this is not a performance issue, be mindful of how your attribute hierarchies, natural hierarchies, and unnatural hierarchies are displayed to end users in your front end tool. For example, if you have a series of geography attributes that are generally queried by using a natural hierarchy of Country/Region, State/Province, and City, you may consider hiding the individual attribute hierarchies for each of these attributes in order to prevent redundancy in the user experience. To hide the attribute hierarchies, use the AttributeHierarchyVisible property.
Maximizing the value of aggregations
An aggregation is a precalculated summary of data that Analysis Services uses to enhance query performance. More specifically, an aggregation summarizes measures by a combination of dimension attributes.
Designing aggregations is the process of selecting the most effective aggregations for your querying workload. As you design aggregations, you must consider the querying benefits that aggregations provide compared with the time it takes to create and refresh the aggregations. On average, having more aggregations helps query performance but increases the processing time involved with building aggregations.
While aggregations are physically designed per measure group partition, the optimization techniques for maximizing aggregation design apply whether you have one or many partitions. In this section, unless otherwise stated, aggregations are discussed in the fundamental context of a cube with a single measure group and single partition. For more information on how you can improve query performance using multiple partitions, see Using partitions to enhance query performance.
How aggregations help
While pre-aggregating data to improve query performance sounds reasonable, how do aggregations actually help Analysis Services satisfy queries more efficiently? The answer is simple. Aggregations reduce the number of records that the Storage Engine needs to scan from disk in order to satisfy a query. To gain some perspective on how this works, first consider how the Storage Engine satisfies a query against a cube with no aggregations.
While you may think that the number of measures and fact table records are the most important factors in aggregating data, dimensions actually play the most critical role in data aggregation, determining how data is summarized in user queries. To help you visualize this, Figure 7 displays three dimensions of a simple sales cube.
Figure 7 Product, Customer, and Order Date dimensions
Each dimension has four attributes. At the grain of the cube, there are 200 individual products, 5,000 individual customers, and 1,095 order dates. The maximum potential number of detailed values in this cube is the Cartesian product of these numbers: 200 * 5000 * 1095 or 109,500,000 theoretical combinations. This theoretical value is only possible if every customer buys every product on every day of every year, which is unlikely. In reality, the data distribution is likely a couple of orders of magnitude lower than the theoretical value. For this scenario, assume that the example cube has 1,095,000 combinations at the grain, a factor of 100 lower than the theoretical value.
Querying the cube at the grain is uncommon, given that such a large result set (1,095,000 cells) is probably not useful for end users. For any query that is not at the cube grain, the Storage Engine must perform an on-the-fly summarization of the detailed cells by the other dimension attributes, which can be costly to query performance. To optimize this summarization, Analysis Services uses aggregations to precalculate and store summaries of data during cube processing. With the aggregations readily available at query time, query performance can be improved greatly.
Continuing with the same cube example, if the cube contains an aggregation of sales by the month and product subcategory attributes, a query that requires sales by month by product subcategory can be directly satisfied by the aggregation without going to the fact data. The maximum number of cells in this aggregation is 720 (20 product subcategory members * 36 months, excluding the All attribute). While the actual number cells in the aggregation is again dependent on the data distribution, the maximum number of cells, 720, is considerably more efficient than summarizing values from 1,095,000 cells.
In addition, the benefit of the aggregation applies beyond those queries that directly match the aggregation. Whenever a query request is issued, the Storage Engine attempts to use any aggregation that can help satisfy the query request, including aggregations that are at a finer level of detail. For these queries, the Storage Engine simply summarizes the cells in the aggregation to produce the desired result set. For example, if you request sales data summarized by month and product category, the Storage Engine can quickly summarize the cells in the month and product subcategory aggregation to satisfy the query, rather than re-summarizing data from the lowest level of detail. To realize this benefit, however, requires that you have properly designed your dimensions with attribute relationships and natural hierarchies so that Analysis Services understands how attributes are related to each other. For more information on dimension design, see Optimizing the dimension design.
How the Storage Engine uses aggregations
To gain some insight into how the Storage Engine uses aggregations, you can use SQL Server Profiler to view how and when aggregations are used to satisfy queries. Within SQL Server Profiler, there are several events that describe how a query is fulfilled. The event that specifically pertains to aggregation hits is the Get Data From Aggregation event. Figure 8 displays a sample query and result set from an example cube.
Figure 8 Sample query and result set
For the query displayed in Figure 8, you can use SQL Server Profiler to compare how the query is resolved in the following two scenarios:
- Scenario 1—Querying against a cube where an aggregation satisfies the query request.
- Scenario 2—Querying against a cube where no aggregation satisfies the query request.
Figure 9 Scenario 1: SQL Server Profiler trace for cube with an aggregation hit
Figure 9 displays a SQL Server Profiler trace of the query’s resolution against a cube with aggregations. In the SQL Server Profiler trace, you can see the operations that the Storage Engine performs to produce the result set.
To satisfy the query, the following operations are performed:
-
After the query is submitted, the Storage Engine gets data from Aggregation C 0000, 0001, 0000 as indicated by the Get Data From Aggregation event.
Aggregation C is the name of the aggregation. Analysis Services assigns the aggregation with a unique name in hexadecimal format. Note that aggregations that have been migrated from earlier versions of Analysis Services use a different naming convention.
- In addition to the aggregation name, Aggregation C, Figure 9 displays a vector, 000, 0001, 0000 , that describes the content of the aggregation. More information on what this vector actually means is described in How to interpret aggregations.
- The aggregation data is loaded into the Storage Engine measure group cache.
- Once in the measure group cache, the Query Execution Engine retrieves the data from the cache and returns the result set to the client.
Figure 10 Scenario 2: SQL Server Profiler trace for cube with no aggregation hit
Figure 10 displays a SQL Server Profiler trace for the same query against the same cube but this time, the cube has no aggregations that can satisfy the query request.
To satisfy the query, the following operations are performed:
- After the query is submitted, rather than retrieving data from an aggregation, the Storage Engine goes to the detail data in the partition.
- From this point, the process is the same. The data is loaded into the Storage Engine measure group cache.
- Once in the measure group cache, the Query Execution Engine retrieves the data from the cache and returns the result set to the client.
To summarize these two scenarios, when SQL Server Profiler displays Get Data From Aggregation, this indicates an aggregation hit. With an aggregation hit, the Storage Engine can retrieve part or all of the data answer from the aggregation and does not need to go to the data detail. Other than fast response times, aggregation hits are a primary indication of a successful aggregation design.
To help you achieve an effective aggregation design, Analysis Services provides tools and techniques to help you create aggregations for your query workload. For more information on these tools and techniques, see Which aggregations are built in this white paper. Once you have created and deployed your aggregations, SQL Server Profiler provides excellent insight to help you monitor aggregation usage over the lifecycle of the application.
Why not create every possible aggregation?
Since aggregations can significantly improve query performance, you may wonder why not create every possible aggregation? Before answering this question, first consider what creating every possible aggregation actually means in theoretical terms.
Note that the goal of this theoretical discussion is to help you understand how aggregations work in an attribute-based architecture. It is not meant to be a discussion of how Analysis Services actually determines which aggregations are built. For more information on this topic, see Which aggregations are built in this white paper.
Generally speaking, an aggregation summarizes measures by a combination of attributes. From an aggregation perspective, for each attribute, there are two levels of detail: the attribute itself and the All attribute. Figure 11 displays the levels of detail for each attribute in the product dimension.
Figure 11 Attribute levels for the product dimension
With four attributes and two levels of detail (All and attribute), the total possible combination for the product dimension is 2*2*2*2 or 2^4 = 16 vectors or potential aggregations.
Figure 12 (3) dimensions with (4) attributes per dimension
If you apply this logic across all attributes displayed in Figure 12, the total number of possible aggregations can be represented as follows:
Total Number of Aggregations
2 (product key) *2 (color) *2 (product subcategory) * 2 (product category)*
2 (customer key) *2 (gender) *2 (city) *2 (state/province) *
2 (order date key) * 2 (month) * 2 (quarter)* 2 (year)
= 2^12
= 4096
Based on this example, the total potential aggregations of any cube can be expressed as 2^ (total number of attributes). While a cube with twelve attributes produces 4,096 theoretical aggregations, a large scale cube may have hundreds of attributes and consequently an exponential increase in the number of aggregations. A cube with 100 attributes, for example, would have 1.26765E+30 theoretical aggregations!
The good news is that this is just a theoretical discussion. Analysis Services only considers a small percentage of these theoretical aggregations, and eventually creates an even smaller subset of aggregations. As a general rule, an effective Analysis Services aggregation design typically contains tens or hundreds of aggregations, not thousands.
With that in mind, the theoretical discussion reminds us that as you add additional attributes to your cube, you are potentially increasing the number of aggregations that Analysis Services must consider. Furthermore, since aggregations are created at the time of cube processing, too many aggregations can negatively impact processing performance or require excessive disk space to store. As a result, ensure that your aggregation design supports the required data refresh timeline.
How to interpret aggregations
When Analysis Services creates an aggregation, each dimension is named by a vector, indicating whether the attribute points to the attribute or to the All level. The Attribute level is represented by 1 and the All level is represented by 0. For example, consider the following examples of aggregation vectors for the product dimension:
- Aggregation By ProductKey Attribute
= [Product Key]:1 [Color]:0 [Subcategory]:0 [Category]:0 or 1000
- Aggregation By Category Attribute
= [Product Key]:0 [Color]:0 [Subcategory]:0 [Category]:1 or 0001
- Aggregation By ProductKey.All
and Color.All and Subcategory.All and Category.All = [Product Key]:0 [Color]:0 [Subcategory]:0 [Category]:0 or 0000
To identify each aggregation, Analysis Services combines the dimension vectors into one long vector path, also called a subcube, with each dimension vector separated by commas.
The order of the dimensions in the vector is determined by the order of the dimensions in the cube. To find the order of dimensions in the cube, use one of the following two techniques. With the cube opened in SQL Server Business Intelligence Development Studio, you can review the order of dimensions in a cube on the Cube Structure tab. The order of dimensions in the cube is displayed in the Dimensions pane, on both the Hierarchies tab and the Attributes tab. As an alternative, you can review the order of dimensions listed in the cube XML file.
The order of attributes in the vector for each dimension is determined by the order of attributes in the dimension. You can identify the order of attributes in each dimension by reviewing the dimension XML file.
For example, the following subcube definition (0000, 0001, 0001) describes an aggregation for:
Product – All, All, All, All
Customer – All, All, All, State/Province
Order Date – All, All, All, Year
Understanding how to read these vectors is helpful when you review aggregation hits in SQL Server Profiler. In SQL Server Profiler, you can view how the vector maps to specific dimension attributes by enabling the Query Subcube Verbose event.
Which aggregations are built
To decide which aggregations are considered and created, Analysis Services provides an aggregation design algorithm that uses a cost/benefit analysis to assess the relative value of each aggregation candidate.
- Aggregation Cost—The cost of an aggregation is primarily influenced by the aggregation size. To calculate the size, Analysis Services gathers statistics including source record counts and member counts, as well as design metadata including the number of dimensions, measures, and attributes. Once the aggregation cost is calculated, Analysis Services performs a series of tests to evaluate the cost against absolute cost thresholds and to evaluate the cost compared to other aggregations.
- Aggregation Benefit—The benefit of the aggregation depends on how well it reduces the amount of data that must be scanned during querying. For example, if you have 1,000,000 data values that are summarized into an aggregation of fifty values, this aggregation greatly benefits query performance. Remember that Analysis Services can satisfy queries by using an aggregation that matches the query subcube exactly, or by summarizing data from an aggregation at a lower level (a more detailed level). As Analysis Services determines which aggregations should be built, the algorithm needs to understand how attributes are related to each other so it can detect which aggregations provide the greatest coverage and which aggregations are potentially redundant and unnecessary.
To help you build aggregations, Analysis Services exposes the algorithm using two tools: the Aggregation Design Wizard and the Usage-Based Optimization Wizard.
- The Aggregation Design Wizard designs aggregations based on your cube design and data distribution. Behind the scenes, it selects aggregations using a cost/benefit algorithm that accepts inputs about the design and data distribution of the cube. The Aggregation Design Wizard can be accessed in either Business Intelligence Development Studio or SQL Server Management Studio.
- The Usage-Based Optimization
Wizard designs aggregations based on query usage patterns. The Usage-Based Optimization Wizard uses the same cost/benefit algorithm as the Aggregation Design Wizard except that it provides additional weighting to those aggregation candidates that are present in the Analysis Services query log. The Usage-Based Optimization Wizard can be accessed in either Business Intelligence Development Studio (BIDS) or SQL Server Management Studio. To use the Usage-Based Optimization Wizard, you must capture end-user queries and store the queries in a query log. To set up and configure the query log for an instance of Analysis Services, you can access a variety of configuration settings in SQL Server Management Studio to control the sampling frequency of queries and the location of the query log.
In those special scenarios when you require finer grained control over aggregation design, SQL Server Service Pack 2 samples includes an advanced Aggregation utility. Using this advanced tool, you can manually create aggregations without using the aggregation design algorithm. For more information on the Aggregation utility, see Appendix C.
How to impact aggregation design
To help Analysis Services successfully apply the aggregation design algorithm, you can perform the following optimization techniques to influence and enhance the aggregation design. (The sections that follow describe each of these techniques in more detail).
Suggesting aggregation candidates – When Analysis Services designs aggregations, the aggregation design algorithm does not automatically consider every attribute for aggregation. Consequently, in your cube design, verify the attributes that are considered for aggregation and determine whether you need to suggest additional aggregation candidates.
Specifying statistics about cube data
– To make intelligent assessments of aggregation costs, the design algorithm analyzes statistics about the cube for each aggregation candidate. Examples of this metadata include member counts and fact table counts. Ensuring that your metadata is up-to-date can improve the effectiveness of your aggregation design.
Adopting an aggregation design strategy – To help you design the most effective aggregations for your implementation, it is useful to adopt an aggregation design strategy that leverages the strengths of each of the aggregation design methods at various stages of your development lifecycle.
Suggesting aggregation candidates
When Analysis Services designs aggregations, the aggregation design algorithm does not automatically consider every attribute for aggregation. Remember the discussion of the potential number of aggregations in a cube? If Analysis Services were to consider every attribute for aggregation, it would take too long to design the aggregations, let alone populate them with data. To streamline this process, Analysis Services uses the Aggregation Usage property to determine which attributes it should automatically consider for aggregation. For every measure group, verify the attributes that are automatically considered for aggregation and then determine whether you need to suggest additional aggregation candidates.
The aggregation usage rules
An aggregation candidate is an attribute that Analysis Services considers for potential aggregation. To determine whether or not a specific attribute is an aggregation candidate, the Storage Engine relies on the value of the Aggregation Usage property. The Aggregation Usage property is assigned a per-cube attribute, so it globally applies across all measure groups and partitions in the cube. For each attribute in a cube, the Aggregation Usage property can have one of four potential values: Full, None, Unrestricted, and Default.
- Full: Every aggregation for the cube must include this attribute or a related attribute that is lower in the attribute chain. For example, you have a product dimension with the following chain of related attributes: Product, Product Subcategory, and Product Category. If you specify the Aggregation Usage for Product Category to be Full, Analysis Services may create an aggregation that includes Product Subcategory as opposed to Product Category, given that Product Subcategory is related to Category and can be used to derive Category totals.
- None—No aggregation for the cube may include this attribute.
- Unrestricted—No restrictions are placed on the aggregation designer; however, the attribute must still be evaluated to determine whether it is a valuable aggregation candidate.
- Default—The designer applies a default rule based on the type of attribute and dimension. As you may guess, this is the default value of the Aggregation Usage property.
The default rule is highly conservative about which attributes are considered for aggregation. Therefore, it is extremely important that you understand how the default rule works. The default rule is broken down into four constraints:
-
Default Constraint 1—Unrestricted for the Granularity and All Attributes – For the dimension attribute that is the measure group granularity attribute and the All attribute, apply Unrestricted. The granularity attribute is the same as the dimension’s key attribute as long as the measure group joins to a dimension using the primary key attribute.
To help you visualize how Default Constraint 1 is applied, Figure 13 displays a product dimension with six attributes. Each attribute is displayed as an attribute hierarchy. In addition, three user hierarchies are included in the dimension. Within the user hierarchies, there are two natural hierarchies displayed in blue and one unnatural hierarchy displayed in grey. In addition to the All attribute (not pictured in the diagram), the attribute in yellow, Product Key, is the only aggregation candidate that is considered after the first constraint is applied. Product Key is the granularity attribute for the measure group.
Figure 13 Product dimension aggregation candidates after applying Default Constraint 1
- Default Constraint 2—None for Special Dimension Types – For all attributes (except All) in many-to-many, nonmaterialized reference dimensions, and data mining dimensions, use None. The product dimension in Figure 13 is a standard dimension. Therefore it is not affected by constraint 2. For more information on many-to-many and reference dimensions, see Complex dimension relationships.
-
Default Constraint 3—Unrestricted for Natural Hierarchies – For all user hierarchies, apply a special scanning process to identify the attributes in natural hierarchies. As you recall, a natural
hierarchy is a user hierarchy where all attributes participating in the hierarchy contain attribute relationships at every level of the hierarchy.
To identify the natural hierarchies, Analysis Services scans each user hierarchy starting at the top level and then moves down through the hierarchy to the bottom level. For each level, it checks whether the attribute of the current level is linked to the attribute of the next level via a direct or indirect attribute relationship, for every attribute that pass the natural hierarchy test, apply Unrestricted, except for nonaggregatable attributes, which are set to Full.
- Default Constraint 4—None For Everything Else. For all other dimension attributes, apply None. In this example, the color attribute falls into this bucket since it is only exposed as an attribute hierarchy.
Figure 14 displays what the Product dimension looks like after all constraints in the default rule have been applied. The attributes in yellow highlight the aggregation candidates.
- As a result of Default Constraint 1, the Product Key and All attributes have been identified as candidates.
- As a result of Default Constraint 3, the Size, Size Range, Subcategory, and Category attributes have also been identified as candidates.
- After Default Constraint 4 is applied, Color is still not considered for any aggregation.
Figure 14 Product dimension aggregation candidates after all application of all default constraints
While the diagrams are helpful to visualize what happens after the each constraint is applied, you can view the specific aggregation candidates for your own implementation when you use the Aggregation Design Wizard to design aggregations.
Figure 15 displays the Specify Object Counts page of the Aggregation Design Wizard. On this Wizard page, you can view the aggregation candidates for the Product dimension displayed in Figure 14. The bold attributes in the Product Dimension are the aggregation candidates for this dimension. The Color attribute is not bold because it is not an aggregation candidate. This Specify Object Counts page is discussed again in Specifying statistics about cube metadata, which describes how you can update statistics to improve aggregation design.
Figure 15 Aggregation candidates in the Aggregation Design Wizard
Influencing aggregation candidates
In light of the behavior of the Aggregation Usage property, following are some guidelines that you can adopt to influence the aggregation candidates for your implementation. Note that by making these modifications, you are influencing the aggregation candidates, not guaranteeing that a specific aggregation is going to be created. The aggregation must still be evaluated for its relative cost and benefit before it is created. The guidelines have been organized into three design scenarios:
- Dimensions with no user hierarchies—If your dimension ONLY has attribute hierarchies with no user-defined hierarchies, by default the only attribute that is considered for aggregation is the cube granularity attribute plus the All attribute. As such, you may want to consider adding some natural hierarchies to your design. In addition to the Aggregation Usage benefits, your users may enjoy the analysis experience of using predefined navigation paths.
-
Attribute only exposed in an attribute hierarchy—If a given attribute is only exposed as an attribute hierarchy such as Color in Figure 14, you may want to change its Aggregation Usage property as follows:
- Change the value of the Aggregation Usage property from Default to Unrestricted if the attribute is a commonly used attribute or if there are special considerations for improving the performance in a particular pivot or drilldown. For example, if you have highly summarized scorecard style reports, you want to ensure that the users experience good initial query response time before drilling around into more detail.
- While setting the Aggregation Usage property of a particular attribute hierarchy to Unrestricted is appropriate is some scenarios, do not be tempted to set all of your attribute hierarchies to Unrestricted. While this scenario seems reasonable, you may quickly find yourself in a position where the Wizard takes a very long time to consider all of the possible candidates and create an aggregation design. In large cubes, the Wizard can take at least an hour to complete the design and considerably much more time to process. As such, you should set the property to Unrestricted only for the commonly queried attribute hierarchies. The general rule is five to ten Unrestricted attributes per dimension.
- Change the value of the Aggregation Usage property from Default to Full in the unusual case that it is used in virtually every query you want to optimize. This is a very rare case and should only be used for attributes that have a relatively small number of members.
- Change the value of the Aggregation Usage property from Default to Unrestricted if the attribute is a commonly used attribute or if there are special considerations for improving the performance in a particular pivot or drilldown. For example, if you have highly summarized scorecard style reports, you want to ensure that the users experience good initial query response time before drilling around into more detail.
- Infrequently used attributes—For attributes participating in natural hierarchies, you may want to change the Aggregation Usage property from Default to None if users would only infrequently use it. Using this approach can help you reduce the aggregation space and get to the five to ten Unrestricted attributes per dimension. For example, you may have certain attributes that are only used by a few advanced users who are willing to accept slightly slower performance. In this scenario, you are essentially forcing the aggregation design algorithm to spend time building only the aggregations that provide the most benefit to the majority of users. Another example where you may want to consider setting the Aggregation Usage property to None is when you have a natural hierarchy where the number of members from one level to the next level is almost identical. For example, if you have 20 product subcategories and 18 product categories, it may make sense to set the product category attribute to None since the I/O penalty of aggregating 20 members up to 18 members is negligible. Generally speaking, if the data does not support at least a 2:1 ratio, you should consider setting the Aggregation Usage to None.
Specifying statistics about cube data
Once the aggregation design algorithm has identified the aggregation candidates, it performs a cost/benefit analysis of each aggregation. In order to make intelligent assessments of aggregation costs, the design algorithm analyzes statistics about the cube for each aggregation candidate. Examples of this metadata include member counts and fact table record counts. Ensuring that your metadata is up-to-date can improve the effectiveness of your aggregation design.
You can define the fact table source record count in the EstimatedRows property of each measure group, and you can define attribute member count in the EstimatedCount property of each attribute.
You can modify these counts in the Specify Counts page of the Aggregation Design Wizard as displayed in Figure 16.
Figure 16 Specify object counts in the Aggregation Design Wizard
If the count is NULL (i.e., you did not define it during design), clicking the Count button populates the counts for each aggregation candidate as well as the fact table size. If the count is already populated, clicking the Count button does not update the counts. Rather you must manually change the counts either in the dialog box or programmatically. This is significant when you design aggregations on a small data set and then move the cube to a production database. Unless you update the counts, any aggregation design is built by using the statistics from the development data set.
In addition, when you use multiple partitions to physically divide your data, it is important that the partition counts accurately reflect the data in the partition and not the data across the measure group. So if you create one partition per year, the partition count for the year attribute should be 1. Any blank counts in the Partition Count column use the Estimated Count values, which apply to the entire fact table.
Using these statistics, Analysis Services compares the cost of each aggregation to predefined cost thresholds to determine whether or not an aggregation is too expensive to build. If the cost is too high, it is immediately discarded. One of the most important cost thresholds is known as the one-third rule. Analysis Services never builds an aggregation that is greater than one third of the size of the fact table. In practical terms, the one-third rule typically prevents the building of aggregations that include one or more large attributes.
As the number of dimension members increases at deeper levels in a cube, it becomes less likely that an aggregation will contain these lower levels because of the one-third rule. The aggregations excluded by the one-third rule are those that would be almost as large as the fact level itself and almost as expensive for Analysis Services to use for query resolution as the fact level. As a result, they add little or no value.
When you have dimensions with a large number of members, this threshold can easily be exceeded at or near the leaf level. For example, you have a measure group with the following design:
- Customer dimension with 5,000,000 individual customers organized into 50,000 sales districts and 5,000 sales territories
- Product dimension with 10,000 products organized into 1,000 subcategories and 30 categories
- Time dimension with 1,095 days organized into 36 months and 3 years
- Sales fact table with 12,000,000 sales records
If you model this measure group using a single partition, Analysis Services does not consider any aggregation that exceeds 4,000,000 records (one third of the size of the partition). For example, it does not consider any aggregation that includes the individual customer, given that the customer attribute itself exceeds the one-third rule. In addition, it does not consider an aggregation of sales territory, category, and month since the total number of records of that aggregation could potentially be 5.4 million records consisting of 5,000 sales territories, 30 categories, and 36 months.
If you model this measure group using multiple partitions, you can impact the aggregation design by breaking down the measure group into smaller physical components and adjusting the statistics for each partition.
For example, if you break down the measure group into 36 monthly partitions, you may have the following data statistics per partition:
- Customer dimension has 600,000 individual customers organized into 10,000 sales districts and 3,000 sales territories
- Product dimension with 7,500 products organized into 700 subcategories and 25 categories
- Time dimension with 1 month and 1 year
- Sales fact table with 1,000,000 sales records
With the data broken down into smaller components, Analysis Services can now identify additional useful aggregations. For example, the aggregation for sales territory, category, and month is now a good candidate with (3000 sales territories *25 categories *1 month) or 75,000 records which is less than one third of the partition size of 1,000,000 records. While creating multiple partitions is helpful, it is also critical that you update the member count and partition count for the partition as displayed in Figure 16. If you do not update the statistics, Analysis Services will not know that the partition contains a reduced data set. Note that this example has been provided to illustrate how you can use multiple partitions to impact aggregation design. For practical partition sizing guidelines, including the recommended number of records per partition, see Designing partitions in this white paper.
Note that you can examine metadata stored on, and retrieve support and monitoring information from, an Analysis Services instance by using XML for Analysis (XMLA) schema rowsets. Using this technique, you can access information regarding partition record counts and aggregation size on disk to help you get a better sense of the footprint of Analysis Services cubes.
Adopting an aggregation design strategy
The goal of an aggregation design strategy is to help you design and maintain aggregations throughout your implementation lifecycle. From an aggregation perspective, the cube lifecycle can be broken down into two general stages: initial aggregation design and ongoing tuning based on query patterns.
Initial Aggregation Design
The most effective aggregation designs are those that are customized for the querying patterns of your user base. Unfortunately, when you initially deploy a cube, you probably will not have query usage data available. As such, it is not possible to use the Usage-Based Optimization Wizard. However, because Analysis Services generally resolves user queries faster with some aggregations than with none, you should initially design a limited number of aggregations by using the Aggregation Design Wizard. The number of initial aggregations that you should design depends on the complexity and size of the cube (the fact size).
- Small cubes—With a small cube, an effective aggregation strategy is to initially design aggregations using the Aggregation Design Wizard to achieve a 20 to 30 percent increase in performance. Note that if the design has too many attributes to consider for aggregation, there is a chance that the Aggregation Design Wizard will stop before reaching the designed percent performance improvement and the user interface may not visibly show any percent performance increase. In this scenario, the large number of attributes has resulted in many possible aggregations, and the number of aggregations that have actually been created are a very small percentage of the total possible aggregations.
- Large and complex cubes—With a large and complex cube, it takes Analysis Services a long time just to design a small percentage of the possible aggregations. Recall that the number of theoretical aggregations in a cube can be expressed as 2^ (total number of Unrestricted attributes). A complex cube with five dimensions that each contain eight attributes has 2^40 or 1.1 trillion aggregation candidates (given that every attribute is an aggregation candidate). With this number, if you assume that the Aggregation Design Wizard can examine 1000 aggregations per second (which is a very generous estimate), it will take the Aggregation Design Wizard approximately 35 years to consider a trillion possible aggregations. Furthermore, a large number of aggregations takes a long time to calculate and consumes a large amount of disk space. While this is a theoretical example, an effective approach with real world cubes that are large and complex is to initially design aggregations to achieve a small performance increase (less than 10 percent and possibly even 1 or 2 percent with very complex cubes) and then allow the Aggregation Design Wizard to run for no more than 15 minutes.
- Medium complexity cubes—With a medium-complexity cube, design aggregations to achieve a 10 to 20 percent increase in performance. Then, allow the wizard to run for no more than 15 minutes. While it is difficult to define what constitutes a high-complexity cube versus a medium-complexity cube, consider this general guideline: a high-complexity cube is a cube contains more than ten Unrestricted attributes in any given dimension.
Before you create initial aggregations with the Aggregation Design Wizard, you should evaluate the application Aggregation Usage property and modify its value as necessary to minimize aggregations that are rarely used and to maximize the probability of useful aggregations. Using Aggregation Usage is equivalent to providing the aggregation algorithm with “hints” about which attributes are frequently and infrequently queried. For specific guidelines on modifying the Aggregation Usage property, see Influencing aggregation candidates.
After you design aggregations for a given partition, it is a good practice to evaluate the size of the aggregation files. The total size of all aggregation files for a given partition should be approximately one to two times the size of the source fact table. If the aggregations are greater than two times the size of the fact table, you are likely spending a long time processing your cube to build relatively large aggregation files. During querying, you can potentially experience performance issues when large aggregation files cannot be effectively loaded into memory due to lack of system resources. If you experience these issues, it is good practice to consider reducing the number of aggregation candidates.
Ongoing tuning based on query patterns
After users have queried the cube for a sufficient period of time to gather useful query pattern data in the query log (perhaps a week or two), use the Usage-Based Optimization Wizard to perform a usage-based analysis for designing additional aggregations that would be useful based on actual user query patterns. You can then process the partition to create the new set of aggregations. As usage patterns change, use the Usage-Based Optimization Wizard to update additional aggregations.
Remember that to use the Usage-Based Optimization Wizard, you must capture end-user queries and store the queries in a query log. Logging queries requires a certain amount of overhead so it is generally recommended that you turn off logging and then turn it back on periodically when you need to tune aggregations based on query patterns.
As an alternative to using the Usage-Based Optimization Wizard, if you require finer grained control over aggregation design, SQL Server Service Pack 2 samples includes an advanced Aggregation Utility that allows you to create specific aggregations from the query log without using the aggregation design algorithm. For more information on the Aggregation Utility, see Appendix C.
Using partitions to enhance query performance
Partitions separate measure group data into physical units. Effective use of partitions can enhance query performance, improve processing performance, and facilitate data management. This section specifically addresses how you can use partitions to improve query performance. The Using partitions to enhance processing performance section discusses the processing and data management benefits of partitions.
How partitions are used in querying
When you query a cube, the Storage Engine attempts to retrieve data from the Storage Engine cache. If no data is available in the cache, it attempts to retrieve data from an aggregation. If no aggregation is present, it must go to the fact data. If you have one partition, Analysis Services must scan though all of the fact data in the partition to find the data that you are interested in. While the Storage Engine can query the partition in parallel and use bitmap indexes to speed up data retrieval, with just one partition, performance is not going to be optimal.
As an alternative, you can use multiple partitions to break up your measure group into separate physical components. Each partition can be queried separately and the Storage Engine can query only the partition(s) that contain the relevant data.
Figure 17 Intelligent querying by partitions
Figure 17 displays a query requesting Reseller Sales Amount by Business Type from a cube called Adventure Works as well as the SQL Server Profiler trace that describes how the query was satisfied. The Reseller Sales measure group of the Adventure Works cube contains four partitions: one for each year. Because the query slices on 2003, the Storage Engine can go directly to the 2003 Reseller Sales partition and does not have to scan data from other partitions. The SQL Server Profiler trace for this query demonstrates how the query needs to read data only from the 2003 Reseller Sales partition.
Designing partitions
If you have some idea how users query the data, you can partition data in a manner that matches common queries. This may be somewhat difficult if user queries do not follow common patterns. A very common choice for partitions is to select an element of time such as day, month, quarter, year or some combination of time elements. Many queries contain a time element, so partitioning by time often benefits query performance.
When you set up your partitions, you must bind each partition to a source table, view, or source query that contains the subset of data for that partition. For MOLAP partitions, during processing Analysis Services internally identifies the slice of data that is contained in each partition by using the Min and Max DataIDs of each attribute to calculate the range of data that is contained in the partition. The data range for each attribute is then combined to create the slice definition for the partition. The slice definition is persisted as a subcube. Knowing this information, the Storage Engine can optimize which partitions it scans during querying by only choosing those partitions that are relevant to the query. For ROLAP and proactive caching partitions, you must manually identify the slice in the properties of the partition.
As you design partitions, use the following guidelines for creating and managing partitions:
- When you decide how to break down your data into partitions, you are generally weighing out partition size vs. number of partitions. Partition size is a function of the number of records in the partition as well as the size of the aggregation files for that partition. Even though the segments in a partition are queried in parallel, if the aggregation files for a partition cannot be effectively managed in memory, you can see significant performance issues during querying.
- In general. the number of records per partition should not exceed 20 million. In addition, the size of a partition should not exceed 250 MB. If the partition exceeds either one of these thresholds, consider breaking the partition into smaller components to reduce the amount of time spent scanning the partition. While having multiple partitions is generally beneficial, having too many partitions, e.g., greater than a few hundred, can also affect performance negatively.
- If you have several partitions that are less than 50 MB or 2 million records per partition, consider consolidating them into one partition. In addition, it is generally not a good practice to create a partition that has less than 4,096 records. In this scenario, given that the record count is so small, the Storage Engine does not create aggregations or indexes for the partition and therefore does not set the auto-slice. Note that this record count threshold is controlled by the IndexBuildThreshold property in the msmdsrv.ini file. Falling below this threshold is generally not an issue in production environments since partition data sets are typically much larger than 4,096 records.
- When you define your partitions, remember that they do not have to contain uniform datasets. For example, for a given measure group, you may have three yearly partitions, 11 monthly partitions, three weekly partitions, and 1–7 daily partitions. The value of using heterogeneous partitions with different levels of detail is that you can more easily manage the loading of new data without disturbing existing partitions. In addition, you can design aggregations for groups of partitions that share the same level of detail (more information on this in the next section).
- Whenever you use multiple partitions for a given measure group, you must ensure that you update the data statistics for each partition. More specifically, it is important to ensure that the partition data and member counts accurately reflect the specific data in the partition and not the data across the entire measure group. For more information on how to update partition counts, see Specifying statistics about cube data.
- For distinct count measure groups, consider specifically defining your partitions to optimize the processing and query performance of distinct counts. For more information on this topic, see Optimizing distinct count.
Aggregation considerations for multiple partitions
For each partition, you can use a different aggregation design. By taking advantage of this flexibility, you can identify those data sets that require higher aggregation design. While the flexibility can definitely help you enhance performance, too many aggregation designs across your partitions can introduce overhead.
To help guide your aggregation design, the following are general guidelines for you to consider. When you have less than ten partitions, you should typically have no more than two aggregation designs per measure group. With less than 50 partitions, you typically want no more than three aggregation designs per measure group. For greater than 50 partitions, you want no more than four aggregation designs.
While each partition can have a different aggregation design, it is a good practice to group your partitions based on the data statistics of the partition so that you can apply a single aggregation design to a group of similar partitions.
Consider the following example. In a cube with multiple monthly partitions, new data may flow into the single partition corresponding to the latest month. Generally that is also the partition most frequently queried. A common aggregation strategy in this case is to perform Usage-Based Optimization to the most recent partition, leaving older, less frequently queried partitions as they are.
The newest aggregation design can also be copied to a base partition. This base partition holds no data—it serves only to hold the current aggregation design. When it is time to add a new partition (for example, at the start of a new month), the base partition can be cloned to a new partition. When the slice is set on the new partition, it is ready to take data as the current partition. Following an initial full process, the current partition can be incrementally updated for the remainder of the period. For more information on processing techniques, see Refreshing partitions efficiently.
Writing efficient MDX
When the Query Execution Engine executes an MDX query, it translates the query into Storage Engine data requests and then compiles the data to produce a query result set.
During query execution, the Query Execution Engine also executes any calculations that are directly or indirectly referenced, such as calculated members, semi-additive measures, and MDX Script scope assignments. Whenever you directly or indirectly reference calculations in your query, you must consider the impact of the calculations on query performance.
This section presents techniques for writing efficient MDX statements in common design scenarios. The section assumes that the reader has some knowledge of MDX.
Specifying the calculation space
When you need to create MDX calculations that apply business rules to certain cells in a cube, it is critical to write efficient MDX code that effectively specifies the calculation space for each rule.
Before you learn about these MDX coding techniques, it is important to be familiar with some common scenarios where conditional business rules are relevant. Following is a description of a growth calculation where you want to apply unique rules to different time periods. To calculate growth from a prior period, you may think that the logical expression for this calculation is current period minus prior period. In general, this expression is valid; however, it is not exactly correct in the following three situations.
- The First Period—In the first time period of a cube, there is no prior period. Since the prior period does not exist for the first period, the expression of current period minus prior period evaluates to current period, which can be somewhat misleading. To avoid end-user confusion, for the prior period, you may decide to apply a business rule that replaces the calculation with NULL.
- The All Attribute—You need to consider how the calculation interacts with the All. In this scenario, the calculation simply does not apply to the All, so you decide to apply a rule that sets the value for All to NA.
- Future Periods—For time periods that extend past the end of the data range, you must consider where you want the calculation to stop. For example, if your last period of data is December of 2006, you want to use a business rule to only apply the calculation to time periods before and including December of 2006.
Based on this analysis, you need to apply four business rules for the growth from prior period calculation. Rule 1 is for the first period, rule 2 is for the All, rule 3 is for the future time periods and rule 4 is for the remaining time periods. The key to effectively applying these business rules is to efficiently identify the calculation space for each rule. To accomplish this, you have two general design choices:
- You can create a calculated member with an IIF statement that uses conditional logic to specify the calculation space for a given period.
- Alternatively you can create a calculated member and use an MDX Script scope assignment to specify the calculation space for a given period.
In a simple cube with no other calculations, the performance of these two approaches is approximately equal. However, if the calculated member directly or indirectly references any other MDX calculations such as semi-additive measures, calculated members, or MDX Script scope assignments, you can definitely see a performance difference between the two approaches. To see how MDX Script scope assignments can be used to perform the growth calculation, you can use the Business Intelligence Wizard to generate time intelligence calculations in your cube.
To better understand the performance benefit of the scope assignment technique used by the Business Intelligence Wizard, consider the following illustrative example of an effective scope assignment. You require a new calculated member called Weekend Gross Profit. The Weekend Gross Profit is derived from the Gross Profit calculated member. To calculate the Weekend Gross Profit, you must sum the Gross Profit calculated member for the days in a sales weekend. This seems easy enough but there are different business rules that apply to each sales territory to as follows:
- For all of the stores in the North America sales territory, the Weekend Gross Profit should sum the Gross Profit for days 5, 6, and 7, i.e., Friday, Saturday, and Sunday.
- For all of the stores in the Pacific sales territory, the Weekend Gross Profit should be NULL. You want to set it to NULL because the Pacific territory is in the process of closing of all of its stores and the Gross Profit numbers are significantly skewed due to their weekend clearance sales.
- For all other territories, the Weekend Gross Profit should sum the Gross Profit for days 6 and 7, i.e., Saturday and Sunday.
To satisfy the conditions of this scenario, following are two design options that you can choose.
Option 1—Calculated Member
You can use a calculated member with an IIF statement to apply the conditional summing of Gross Profit. In this scenario, the Query Execution Engine must evaluate the calculation space at runtime on a cell-by-cell basis based on the IIF specification. As a result, the Query Execution Engine uses a less optimized code path to execute the calculation that increases query response time.
with member [Option 1 Weekend Gross Profit] as
iif (ancestor([Sales Territory].[Sales Territory].CurrentMember,
[Sales Territory].[Sales Territory].[Group])
IS [Sales Territory].[Sales Territory].[Group].&[North America],
Sum({[Date].[Day of Week].&[5],
[Date].[Day of Week].&[6],
[Date].[Day of Week].&[7]},
[Measures].[Reseller Gross Profit]),
iif (ancestor([Sales Territory].[Sales Territory].CurrentMember,
[Sales Territory].[Sales Territory].[Group])
IS [Sales Territory].[Sales Territory].[Group].&[Pacific],
NULL,
Sum({[Date].[Day of Week].&[6],
[Date].[Day of Week].&[7]},
[Measures].[Reseller Gross Profit])))
Option 2—Scope assignment with a Calculated Member
In Option 2, you use a calculated member with a scope assignment to apply the conditional summing of Gross Profit.
CREATE MEMBER CURRENTCUBE.[MEASURES].[Option 2 Weekend Gross Profit] AS
Sum({[Date].[Day of Week].&[7],
[Date].[Day of Week].&[6] },
[Measures].[Reseller Gross Profit]);
Scope ([Option 2 Weekend Gross Profit],
Descendants([Sales Territory].[Sales Territory Group].&[North
America]));
This = Sum({[Date].[Day of Week].&[7],
[Date].[Day of Week].&[6],
[Date].[Day of Week].&[5] },
[Measures].[Reseller Gross Profit]);
End Scope;
Scope ([Option 2 Weekend Gross Profit],
Descendants([Sales Territory].[Sales Territory Group].&[Pacific]));
This = NULL;
End Scope;
In this example, this option is significantly faster than the first option. The reason this option is faster is because the two scope subcube definitions (the left hand side of each scope assignment) enable the Query Execution Engine to know ahead of time the calculation space for each business rule. Using this information, the Query Execution Engine can select an optimized execution path to execute the calculation on the specified range of cells. As a general rule, it is a best practice to always try to simplify calculation expressions by moving the complex parts into multiple Scope definitions whenever possible.
Note that in this Scope assignment, the Descendants function represents the Analysis Services 2000 approach to using MDX functions to navigate a dimension hierarchy. An alternative approach in Analysis Services 2005 is to simply use Scope with the attribute hierarchies. So instead of using Scope (Descendants([Sales Territory].[Pacific])), you can use Scope ([Sales Territory].[Pacific]).
To enhance performance, wherever possible, use the scope subcube definition to narrowly define the calculation space. While this is the ideal scenario, there are situations where it is not possible. The most common scenario is when you need to define a calculation on an arbitrary collection of cells ([Measures].[Reseller Sales Amount] >500). This type of expression is not allowed in the scope subcube definition since the definition must be static. In this scenario, the solution is to define a broader calculation space in the scope definition, and then use the scope MDX expression (the right hand side of the scope assignment) to narrow down the cube space using the IIF statement. The following is an example of how this statement can be structured to apply a weighting factor to the North American sales:
SCOPE ([Measures].[Sales Amount],
[Sales Territory].[Sales Territory].[Group].&[North America])
THIS =
IIF ([Measures].[Sales Amount] > 1000, [Measures].[Sales Amount],
[Measures].[Sales Amount]*1.2);
END SCOPE
From a performance perspective, MDX scope assignments provide an efficient alternative to using IIF to apply unique business rules to certain cells in a cube. Whenever you need to conditionally apply calculations, you should consider this approach.
Removing empty tuples
When you write MDX statements that use a set function such as Crossjoin, Descendants, or Members, the default behavior of the function is to return both empty and nonempty tuples. From a performance perspective, empty tuples can not only increase the number of rows and/or columns in your result set, but they can also increase query response time.
In many business scenarios, empty tuples can be removed from a set without sacrificing analysis capabilities. Analysis Services provides a variety of techniques to remove empty tuples depending on your design scenario.
Analysis Services ground rules for interpreting null values
Before describing these techniques, it is important to establish some ground rules about how empty values are interpreted in various design scenarios.
- Missing fact table records—Earlier in the white paper, it was stated that only records that are present in the relational fact table are stored in the partition. For example, a sales fact table only stores records for those customers with sales for a particular product. If a customer never purchased a product, a fact table record will not exist for that customer and product combination. The same holds true for the Analysis Services partition. If a fact table record does not exist for a particular combination of dimension members, the cube cells for these dimension members are considered empty.
- Null values in measures—For each fact table measure that is loaded into the partition, you can decide how Analysis Services interprets null values. Consider the following example. Your sales fact table contains a record that has a sales amount of 1000 and a discount amount of null. When discount is loaded into the cube, by default it is interpreted as a zero, which means that it is not considered empty. How Analysis Services interprets null values is controlled by a property called NullProcessing. The NullProcessing property is set on a measure-by-measure basis. By default, it is set to Automatic which means that Analysis Services converts the null values to zero. If you want to preserve the null value from the source system, such as in the example of the discount measure, configure the NullProcessingProperty of that measure to Preserve instead of Automatic.
- Null values in calculations—In calculations, it is important to understand how Nulls are evaluated. For example, 1 minus Null equals 1, not Null. In this example, the Null is treated like a zero for calculation purposes. which may or may not be what you want to use. To explicitly test whether a tuple is null, use the ISEmpty function within an IIF statement to conditionally handle empty tuples.
-
Empty members—When writing calculations that reference dimension members, you may need to handle scenarios where specific members do not exist, such as the parent of the All. In this scenario, the ISEmpty function is not appropriate as it tests for empty cells. Rather in this scenario you want to use the IS operator to test whether the member IS NULL.
General techniques for removing empty tuples
The following describes the most general techniques for removing empty tuples:
-
Non Empty Keyword—When you want to remove empty rows or columns from the axes of an MDX query, you can use the NON EMPTY keyword. Most client applications use the NON EMPTY keyword to remove empty cells in query result sets. The NON EMPTY keyword is applied to an axis and takes effect after the query result set is determined, i.e., after the Query Execution Engine completes the axis tuples with the current members taken from the rows axis, columns axis, and WHERE clause (as well as the default members of the attribute hierarchies not referenced in the query).
Consider the following example displayed in Figure 18. Note that only a subset of the query results is shown.
Figure 18 – Query without Non Empty Keyword
In Figure 18, the rows axis returns a complete list of resellers, regardless of whether or not they have sales in 2003. For each reseller, the columns axis displays the reseller’s 2003 sales. If the reseller had no sales in 2003, the reseller is still returned in the list, it is just returned with a (null) value.
To remove resellers who do not have sales in 2003, you can use the NON EMPTY keyword as displayed in Figure 19. Note that only a subset of the query results is shown.
Figure 19 – Query with Non Empty Keyword
In Figure 19, the NON EMPTY keyword removes the resellers that do not have sales, i.e., null sales for 2003. To apply the NON EMPTY keyword, the Query Execution Engine must completely evaluate all cells in the query before it can remove the empty tuples.
If the query references a calculated member, the Query Execution Engine must evaluate the calculated member for all cells in the query and then remove the empty cells. Consider the example displayed in Figure 20. Note that only a subset of the query results is shown. In this example, you have modified the reseller query, replacing Reseller Sales Amount with the calculated measure Prior Year Variance. To produce the query result set, Analysis Services first obtains a complete set of resellers and then removes those resellers that have an empty prior year variance.
Figure 20 – Query with Non Empty Keyword and Calculated Measure
Given the ground rules about null interpretation, it is necessary to point out when the prior year variance is going to be null.
- Acceptable Sales & Service has no sales in 2003 but has sales of $838.92 in 2002. The Prior Year Variance calculation is null minus $838.92 which evaluates to -$838.92. Since the value is not null, that reseller is returned in the result set.
- Accessories Network has $729.36 sales in 2003 but no sales in 2002. The Prior Year Variance calculation is $729.36 minus null which evaluates to $729.36. Since the value is not null, the reseller is returned in the result set.
- If a reseller has no sales in either 2003 or 2002, the calculation is null minus null, which evaluates to null. The reseller is removed from the result set.
-
NonEmpty Function—The NonEmpty function, NonEmpty(), is similar to the NON EMPTY keyword but it provides additional flexibility and more granular control. While the NON EMPTY keyword can only be applied to an axis, the NonEmpty function can be applied to a set. This is especially useful when you are writing MDX calculations.
As an additional benefit, the NonEmpty function allows you use an MDX expression to evaluate the empty condition against a business rule. The business rule can reference any tuple, including calculated members. Note that if you do not specify the MDX expression in the NonEmpty function, the NonEmpty function behaves just like the NON EMPTY keyword, and the empty condition is evaluated according to the context of the query.
Continuing with the previous examples regarding resellers, you want to change the previous queries to apply your own business rule that decides whether or not a reseller is returned in the query. The business rule is as follows. You want a list of all resellers that had a sale in 2002. For each of these resellers, you only want to display their sales for 2003. To satisfy this requirement, you use the query displayed in Figure 21. Note that only a subset of the query results is shown.
Figure 21 Query with NonEmpty Function
In this query, the NonEmpty function returns those resellers that had a sale in 2002. For each of these resellers, their 2003 sales are displayed. Note that query produces a different list of resellers than the resellers returned in the NON EMPTY keyword example. The Accessories Network reseller has been removed because it only has sales in 2003 with no sales in 2002.
Note that an alternative way to write this query is to use the FILTER expression, such as the following:
SELECT FILTER ([Reseller].[Reseller].[Reseller].Members,
NOT ISEmpty(([Measures].[Reseller Sales Amount],
[Date].[Calendar Year].&[2002]))) on rows,
[Measures].[Reseller Sales Amount] on columns
FROM [Adventure Works]
WHERE [Date].[Calendar Year].&[2003]
In this query, FILTER is used to return only those resellers who had a sale in 2002. FILTER was commonly used in prior versions of Analysis Services. For simple expressions like the one depicted in the above example, Analysis Services actually re-writes the query behind-the-scenes using the NonEmpty function. For other more complicated expressions, it is advisable to use the NonEmpty function in place of the FILTER expression. In Analysis Services 2005, the NonEmpty function provides a more optimized alternative to using the FILTER expression to check for empty cells.
As you use NON EMPTY keyword and NonEmpty function to remove empty tuples, consider the following guidelines:
- The NonEmpty function and NON EMPTY keyword will have approximately the same performance when the parameters passed to NonEmpty function coincide with the query axes of a NON EMPTY keyword query.
- In common scenarios, Non Empty is normally used instead of NonEmpty() at the top level of SELECT query axes.
- In calculations or query sub expressions, NonEmpty() is the recommended approach to achieve similar semantics. When using NonEmpty(), pay extra attention to making sure that the current cell context used by NonEmpty() is the intended one, possibly by specifying additional sets and members as the second parameter of the function.
-
Non_Empty_Behavior (NEB)—Whether or not an expression resolves to null is important for two major reasons. First, most client applications use the NON EMPTY key word in a query. If you can tell the Query Execution Engine that you know an expression will evaluate to null, it does not need to be computed and can be eliminated from the query results before the expression is evaluated. Second, the Query Execution Engine can use the knowledge of a calculation’s Non_Empty_Behavior (NEB) even when the NON EMPTY keyword is not used. If a cell’s expression evaluates to null, it does not have to be computed during query evaluation.
Note that the current distinction between how the Query Execution Engine uses an expression’s NEB is really an artifact of the Query Execution Engine design. This distinction indicates whether one or both optimizations are used, depending how the NEB calculation property is defined. The first optimization is called NonEmpty Optimization and the second optimization is called Query Execution Engine Optimization.
When an expression’s NEB is defined, the author is guaranteeing that the result is null when the NEB is null and consequently the result set is not null when NEB is not null. This information is used internally by the Query Execution Engine to build the query plan.
To use NEB for a given calculation, you provide an expression that defines the conditions under which the calculation is guaranteed to be empty. The reason why NEB is an advanced setting is because it is often difficult to correctly identify the conditions under which the calculation is guaranteed to be empty. If you incorrectly set NEB, you will receive incorrect calculation results. As such, the primary consideration of using NEB is to ensure first and foremost that you have defined the correct expression, before taking into account any performance goals.
The NEB expression can be a fact table measure, a list of two or more fact table measures, a tuple, or a single measure set. To help you better understand which optimizations are used for each expression; consider the guidelines in Table 2.
Table 2 NEB guidelines
|
Calculation Type |
NEB |
Query Execution Engine Optimization Support |
NonEmpty Optimization Support |
Example |
|
Calculated Measure |
Constant measure |
Yes |
Yes |
With Member Measures.DollarSales As Measures.Sales / Measures.ExchangeRate, NEB = Measures.Sales |
|
Calculated Measure |
(List of two or more constant measure references) |
No |
Yes |
With Member Measures.Profit As Measures.Sales – Measures.Cost, NEB = {Measures.Sales, Measures.Cost} |
|
Any (calculated member, script assignment, calculated cell) |
Constant tuple reference Constant single-measure set |
Yes |
No |
Scope [Measures].[store cost]; This = iif( [Measures].[Exchange Rate]>0, [Measures].[Store Cost]/[Measures].[Exchange Rate], null ); Non_Empty_Behavior(This) = [Measures].[Store Cost]; End Scope; |
In addition to understanding the guidelines for the NEB expression, it is important to consider how the expression is applied for various types of calculation operations.
-
Scenario 1—Addition or Subtraction: Example – Measures.M1 + or – Measures.M2. The following guidelines apply for addition and subtraction:
- In general, you MUST specify both measures in the NEB expression for correctness reasons.
- In particular, if both measures belong to the same measure group, it may be possible to specify just one of them in NEB expression if the data supports it. This could result in better performance.
- In general, you MUST specify both measures in the NEB expression for correctness reasons.
-
Scenario 2—Multiplication. Example – Measures.M1 * Measures.M2. The following guidelines apply for multiplication:
- In general, you CANNOT specify any correct NEB expression for this calculation.
- In particular, if it is guaranteed that one of the measures is never null (e.g., a currency exchange rate), you MAY specify the other measure in NEB expression.
- In particular, if it is guaranteed that, for any given cell, either both measures are null, or both are non-null (e.g. they belong to the same measure group), you MAY specify both measures in the NEB expression, OR specify a single measure.
- In general, you CANNOT specify any correct NEB expression for this calculation.
- Scenario 3—Division. Example – Measures.M1 / Measures.M2. In this scenario, you MUST specify the first measure (the numerator, M1) in NEB.
Evaluating empty by using a measure group
In some design scenarios, you may be able to optimize the removal of empty tuples by evaluating the empty condition against an entire measure group. In other words, if a tuple corresponds to a fact data record in the measure group, the tuple is included. If the tuple does not have a fact data record, it is excluded. To apply this syntax, you can use a special version of the Exists function, Exists (Set,, “Measure Group”). Note that this special version of the Exists function actually behaves very differently from the regular Exists function and includes a third parameter of type string where you can specify the name of the desired measure group.
While this approach can be very powerful in removing empty tuples, you must evaluate whether it generates the correct result set. Consider the following example. The sales fact table contains a record corresponding to a reseller’s purchase. In this record, Sales Amount has a value of 500 while Discount Amount is null. To write a query where you only see the resellers with discounts, you cannot use this approach since the reseller still exists in measure group whether or not the reseller’s Discount Amount is null. To satisfy this query, you will need to use the NON EMPTY keyword or NonEmpty function as long as you have properly configured the Null Processing property for the measures in that measure group.
When you do have scenarios where you can apply the Exists function, keep in mind that the Exists function ignores all calculated members. In addition, note that the Exists function used with a measure group specification replaces the deprecated NonEmptyCrossJoin function, which was used in prior versions of Analysis Services to achieve similar functionality.
Removing empty member combinations
Analysis Services 2005 provides a rich attribute architecture that allows you to analyze data across multiple attributes at a given time. When you write queries that involve multiple attributes, there are some optimizations that you should be aware of to ensure that the queries are efficiently evaluated.
- Autoexists—Autoexists is applied behind the scenes whenever you use the Crossjoin function to cross join two attribute hierarchies from the same dimension or, more broadly speaking, whenever you Crossjoin sets with common dimensionality. Autoexists retains only the members that exist with each other so that you do not see empty member combinations that never occur such as (Seattle, Scotland). In some design scenarios, you may have a choice as to whether you create two attributes in a given dimension or model them as two separate dimensions. For example, in an employee dimension you may consider whether or not you should include department in that dimension or add department as another dimension. If you include department in the same dimension, you can take advantage of Autoexists to remove empty combinations of employees and departments.
- Exists function—Using the Exists function in the form of Exists (Set1, Set2), you can remove tuples from one set that do not exist in another set by taking advantage of Autoexists. For example, Exists (Customer.Customer.Members, Customer.Gender.Male) only returns male customers.
- EXISTING operator—The EXISTING operator is similar to the Exists function but it uses as the filter set, the current coordinate specified in your MDX query. Since it uses the current coordinate, it reflects whatever you are slicing by in your query.
In Figure 22, a calculated measure counts a set of customers defined by the EXISTING operator and the WHERE clause of the query slices on Male customers. The result set of this query is a total count of male customers.
Figure 22 Calculated Measure Using Existing Operator
Summarizing data with MDX
Analysis Services naturally aggregates measures across the attributes of each dimension. While measures provide tremendous analytical value, you may encounter scenarios when you want to perform additional aggregations of data, either aggregating MDX calculations or aggregating subsets of the cube that satisfy specific business rules.
Consider the following examples where you may need to aggregate data in MDX:
- Performing time series analysis—You need
to sum reseller profit for all time periods in this year up to and include the current time period.
- Aggregating custom sets—You need to average year-over-year sales variance across all resellers in the USA who sold more than 10,000 units.
- Aggregating calculations at dimension leaves—You need to aggregate the hours worked by each employee multiplied by the employee’s hourly rate.
While using MDX calculated measures with functions like Sum, Average, and Aggregate is a valid approach to summarizing data in these scenarios, from a performance perspective, summarizing data through MDX is not a trivial task and can potentially result in slow performance in large-scale cubes or cubes with many nested MDX calculations. If you experience performance issues summarizing data in MDX, you may want to consider the following design alternatives:
Create a named calculation in the data source view
Depending on the scenario, you should first consider whether you need to perform additional aggregations in MDX or whether you can take advantage of the natural aggregation of the cube. Consider the Profit Margin calculation. Profit is defined by Revenue minus Cost. Assuming that these two measures are stored in the same fact table, instead of defining a calculated member that is calculated on the fly, you can move the Profit calculation to a measure. In the data source view you can create a named calculation on the fact table that defines Profit as Revenue minus Cost. Then, you can add the named calculation as a measure in your cube to be aggregated just like every other measure.
Generally speaking, any time that you have a calculated member that performs addition and subtraction from columns on the same fact table, you can move the calculation to measure. If you do this, keep in mind that SQL operations on NULL data are not identical to MDX. In addition, even though you can add the calculation to the fact table, you must also evaluate the impact of additional measures on the cube. Whenever you query a measure group, even if you only request one measure, all measures in that measure group are retrieved from the Storage Engine and loaded into the data cache. The more measures you have, the greater the resource demands. Therefore, it is important to evaluate the performance benefits on a case by case basis.
Use measure expressions
Measure expressions are calculations that the Storage Engine can perform. Using
measure expressions, you can multiply or divide data from two different measure groups at the measure group leaves and then aggregate the data as a part of normal cube processing. The classic example of this is when you have a weighting factor stored in one measure group, such as currency rates, and you want to apply that to another measure group such as sales. Instead of
creating an MDX calculation that aggregates the multiplication of these two measures at the measure group leaves, you can use measure expressions as an optimized solution. This kind of calculation is perfect for a measure expression since it is somewhat more difficult to accomplish in the data source view given that the measures are from different source fact tables and likely have different granularities. To perform the calculation in the data source view, you can use a named query to join the tables; however, the measure expression typically provides a more efficient solution.
While measure expressions can prove to be very useful, note that when you use a measure expression, the Storage Engine evaluates the expression in isolation of the Query Execution Engine. If any of the measures involved in the measure expression depend on MDX calculations, the Storage Engine evaluates the measure expressions without any regard for the MDX calculations, producing an incorrect result set. In this case, rather than using a measure expression, you can use a calculated member or scope assignment.
Use semiadditive measures and unary operators
Instead of writing complex MDX calculations to handle semiadditive measures, you can use the semiadditive aggregate functions like FirstChild, LastChild, etc. Note that semiadditive functions are a feature of SQL Server Enterprise Edition. In addition to using semiadditive measures, in finance applications, instead of writing complicated MDX expressions that apply custom aggregation logic to individual accounts, you can use unary operators with parent-child hierarchies to apply a custom roll up operator for each account. Note that while parent-child hierarchies are restricted in their aggregation design, they can be faster and less complex than custom MDX. For more information on parent-child hierarchies, see Parent-child hierarchies in this white paper.
Move numeric attributes to measures
You can
convert numeric attributes to measures whenever you have an attribute such as Population or Salary that you need to aggregate. Instead of writing MDX expressions to aggregate these values, consider defining a separate measure group on the dimension table containing the attribute and then defining a measure on the attribute column. So for example, you can replace Sum(Customer.City.Members, Customer.Population.MemberValue) by adding a new measure group on the dimension table with a Sum measure on the Population column.
Aggregate subsets of data
In many scenarios, you want to aggregate subsets of data that meet specific business rules. Before you write a complex filter statement to identify the desired set, evaluate whether you can substitute a filter expression by using Crossjoin or Exists with specific members of your attribute hierarchies.
Consider the following examples.
-
If you want a set of resellers with 81 to 100 employees and you have the number of employees stored in a separate attribute, you can easily satisfy this request with the following syntax:
Exists([Reseller].[Reseller].members,
[Reseller].[Number of Employees].[81]:
[Reseller].[Number of Employees].[100])
In this example, you use Exists with a range for the Number of Employees attribute hierarchy to filter resellers in the 81–100 employee range. The value of this approach is that you can arbitrarily set the ranges based on user requests.
-
If your reseller dimension is large and the ranges that you need to calculate are fixed and commonly used across all users, you can pre-build an attribute that groups the resellers according to employee size ranges. With this new attribute, the previous statement could be written as follows:
Exists([Reseller].[Reseller].members,
[Reseller].[Employee Size Range].[81 to 100])
-
Now if you want the sum of the gross profit for resellers with 81 to 100 employees, you can satisfy this request with the following solutions.
For the range of values, you can use the following syntax.
Sum([Reseller].[Number of Employees].[81]:
[Reseller].[Number of Employees].[100],
[Measures].[Reseller Gross Profit])
For the custom range attribute, you can use the following syntax.
([Reseller].[Number of Employees].[81 – 100],
[Measures].[Reseller Gross Profit])
Note that in both of these solutions, the set of resellers is not necessary.
Using an attribute hierarchy to slice data sets provides a superior solution to aggregating a set filtered on member property values, which was a common practice in prior versions of Analysis Services. Retrieving a member’s properties can be slow since each member needs to be retrieved as well as its property value. With attribute hierarchies, you can leverage the normal aggregation of the cube.
Taking advantage of the Query Execution Engine cache
During the execution of an MDX query, the Query Execution Engine stores calculation results in the Query Execution Engine cache. The primary benefits of the cache are to optimize the evaluation of calculations and to support the re-usage of calculation results across users. To understand how the Query Execution Engine uses calculation caching during query execution, consider the following example. You have a calculated member called Profit Margin. When an MDX query requests Profit Margin by Sales Territory, the Query Execution Engine caches the Profit Margin values for each Sales Territory, assuming that the Sales Territory has a nonempty Profit Margin. To manage the re-usage of the cached results across users, the Query Execution Engine uses scopes. Note that Query Execution scopes should not be confused with the SCOPE keyword in an MDX script. Each Query Execution Engine scope maintains its own cache and has the following characteristics.
- Query
Scope—The query scope contains any calculations created within a query by using the WITH keyword. The query scope is created on demand and terminates when the query is over. Therefore, the cache of the query scope is not shared across queries in a session.
- Session
Scope—The session scope contains calculations created in a given session by using the CREATE statement. The cache of the session scope is reused from request to request in the same session, but is not shared across sessions.
- Global Scope—The global scope contains the MDX script, unary operators, and custom rollups for a given set of security permissions. The cache of the global scope can be shared across sessions if the sessions share the same security roles.
The scopes are tiered in terms of their level of re-usage. The query scope is considered to be the lowest scope, because it has no potential for re-usage. The global scope is considered to be the highest scope, because it has the greatest potential for re-usage.
During execution, every MDX query must reference all three scopes to identify all of the potential calculations and security conditions that can impact the evaluation of the query. For example, if you have a query that contains a query calculated member, to resolve the query, the Query Execution Engine creates a query scope to resolve the query calculated member, creates a session scope to evaluate session calculations, and creates a global scope to evaluate the MDX script and retrieve the security permissions of the user who submitted the query. Note that these scopes are created only if they aren’t already built. Once they are built for a session, they are usually just re-used for subsequent queries to that cube.
Even though a query references all three scopes, it can only use the cache of a single scope. This means that on a per-query basis, the Query Execution Engine must select which cache to use. The Query Execution Engine always attempts to use the cache of the highest possible scope depending on whether or not the Query Execution Engine detects the presence of calculations at a lower scope.
If the Query Execution Engine detects any calculations that are created at the query scope, it always uses the query scope cache, even if a query references calculations from the global scope. If there are no query-scope calculations, but there are session-scope calculations, the Query Execution Engine uses the cache of session scope. It does not matter whether or not the session calculations are actually used in a query. The Query Execution Engine selects the cache based on the presence of any calculation in the scope. This behavior is especially relevant to users with MDX-generating front-end tools. If the front-end tool creates any session scoped calculations, the global cache is not used, even if you do not specifically use the session calculation in a given query.
There are other calculation scenarios that impact how the Query Execution Engine caches calculations. When you call a stored procedure from an MDX calculation, the engine always uses the query cache. This is because stored procedures are nondeterministic. This means that there is no guarantee of what the stored procedure will return. As a result, nothing will be cached globally or in the session cache. Rather, the calculations will only be stored in the query cache. In addition, the following scenarios determine how the Query Execution Engine caches calculation results:
- If you enable visual totals for the session by setting the default MDX Visual Mode property of the Analysis Services connection string to 1, the Query Execution Engine uses the query cache for all queries issued in that session.
- If you enable visual totals for a query by using the MDX VisualTotals function, the Query Execution Engine uses the query cache.
- Queries that use the subselect syntax (SELECT FROM SELECT) or are based on a session subcube (CREATE SUBCUBE) could cause the query cache to be used.
- Arbitrary shape sets can only use the query cache when they are used in a subselect, in the WHERE clause, or in a calculated member Aggregate expression referenced in the WHERE clause. An example of arbitrary shape set is a set of multiple members from different levels of a parent-child hierarchy.
Based on this behavior, when your querying workload can benefit from re-using data across users, it is a good practice to define calculations in the global scope. An example of this scenario is a structured reporting workload where you have a few security roles. By contrast, if you have a workload that requires individual data sets for each user, such as in an HR cube where you have a many security roles or you are using dynamic security, the opportunity to re-use calculation results across users is lessened and the performance benefits associated with re-using the Query Execution Engine cache are not as high.
Applying calculation best practices
While many MDX recommendations must be evaluated in the context of a design scenario, the following best practices are optimization techniques that apply to most MDX calculations regardless of the scenario.
Use the Format String property
Instead of applying conditional logic to return customized values if the cell is EMPTY or 0, use the Format String property. The Format String property provides a mechanism to format the value of a cell. You can specify a user-defined formatting expression for positive values, negative values, zeros, and nulls. The Format String display property has considerably less overhead than writing a calculation or assignment that must invoke the Query Execution Engine. Keep in mind that your front-end tool must support this property.
Avoid late-binding functions
When writing MDX calculations against large data sets involving multiple iterations, avoid referencing late binding functions whose metadata cannot be evaluated until run time. Examples of these functions include: LinkMember, StrToSet, StrToMember, StrToValue, and LookupCube. Because they are evaluated at run time, the Query Execution Engine cannot select the most efficient execution path.
Eliminate redundancy
When you use a function that has default arguments such as Time.CurrentMember, you can experience performance benefits if you do not redundantly specify the default argument. For example, use PeriodsToDate([Date].[Calendar].[Calendar Year]) instead of PeriodsToDate([Date].[Calendar].[Calendar Year], [Date].Calendar.CurrentMember). To take advantage of this benefit, you must ensure that you only have one default Time Hierarchy in your application. Otherwise, you must explicitly specify the member in your calculation.
Ordering expression arguments
When writing calculation expressions like “expr1 * expr2”, make sure the expression sweeping the largest area/volume in the cube space and having the most Empty (Null) values is on the left side. For instance, write “Sales * ExchangeRate” instead of “ExchangeRate * Sales”, and “Sales * 1.15” instead of “1.15 * Sales”. This is because the Query Execution Engine iterates the first expression over the second expression. The smaller the area in the second expression, the fewer iterations the Query Execution Engine needs to perform, and the faster the performance.
Use IS
When you need to check the value of a member, use IIF [Customer].[Company] IS [Microsoft] and not IIF [Customer].[Company].Name = “Microsoft”. The reason that IS is faster is because the Query Execution Engine does not need to spend extra time translating members into strings.
Tuning Processing Performance
Processing is the general operation that loads data from one or more data sources into one or more Analysis Services objects. While OLAP systems are not generally judged by how fast they process data, processing performance impacts how quickly new data is available for querying. While every application has different data refresh requirements, ranging from monthly updates to “near real-time” data refreshes, the faster the processing performance, the sooner users can query refreshed data.
Note that “near real-time” data processing is considered to be a special design scenario that has its own set of performance tuning techniques. For more information on this topic, see Near real-time data refreshes.
To help you effectively satisfy your data refresh requirements, the following provides an overview of the processing performance topics that are discussed in this section:
Understanding the processing architecture – For readers unfamiliar with the processing architecture of Analysis Services, this section provides an overview of processing jobs and how they apply to dimensions and partitions. Optimizing processing performance requires understanding how these jobs are created, used, and managed during the refresh of Analysis Services objects.
Refreshing dimensions efficiently – The performance goal of dimension processing is to refresh dimension data in an efficient manner that does not negatively impact the query performance of dependent partitions. The following techniques for accomplishing this goal are discussed in this section: optimizing SQL source queries, reducing attribute overhead, and preparing each dimension attribute to efficiently handle inserts, updates, deletes as necessary.
Refreshing partitions efficiently – The performance goal of partition processing is to refresh fact data and aggregations in an efficient manner that satisfies your overall data refresh requirements. The following techniques for accomplishing this goal are discussed in this section: optimizing SQL source queries, using multiple partitions, effectively handling data inserts, updates, and deletes, and evaluating the usage of rigid vs. flexible aggregations.
Understanding the processing architecture
Processing is typically described in the simple terms of loading data from one or more data sources into one or more Analysis Services objects. While this is generally true, Analysis Services provides the ability to perform a broad range of processing operations to satisfy the data refresh requirements of various server environments and data configurations.
Processing job overview
To manage processing operations, Analysis Services uses centrally controlled jobs. A processing job is a generic unit of work generated by a processing request. Note that while jobs are a core component of the processing architecture, jobs are not only used during processing. For more information on how jobs are used during querying, see Job architecture.
From an architectural perspective, a job can be broken down into parent jobs and child jobs. For a given object, you can have multiple levels of nested jobs depending on where the object is located in the database hierarchy. The number and type of parent and child jobs depend on 1) the object that you are processing such as a dimension, cube, measure group, or partition, and 2) the processing operation that you are requesting such as a ProcessFull, ProcessUpdate, or ProcessIndexes. For example, when you issue a ProcessFull for a measure group, a parent job is created for the measure group with child jobs created for each partition. For each partition, a series of child jobs are spawned to carry out the ProcessFull of the fact data and aggregations. In addition, Analysis Services implements dependencies between jobs. For example, cube jobs are dependent on dimension jobs.
The most significant opportunities to tune performance involve the processing jobs for the core processing objects: dimensions and partitions.
Dimension processing jobs
During the processing of MOLAP dimensions, jobs are used to extract, index, and persist data in a series of dimension stores. For more information on the structure and content of the dimension stores, see Data retrieval: dimensions. To create these dimension stores, the Storage Engine uses the series of jobs displayed in Figure 23.
Figure 23 Dimension processing jobs
Build attribute stores
For each attribute in a dimension, a job is instantiated to extract and persist the attribute members into an attribute store. As stated earlier, the attribute store primarily consists of the key store, name store, and relationship store. While Analysis Services is capable of processing multiple attributes in parallel, it requires that an order of operations must be maintained. The order of operations is determined by the attribute relationships in the dimension. The relationship store defines the attribute’s relationships to other attributes. In order for this store to be built correctly, for a given attribute, all dependent attributes must already be processed before its relationship store is built. To provide the correct workflow, the Storage Engine analyzes the attribute relationships in the dimension, assesses the dependencies among the attributes, and then creates an execution tree that indicates the order in which attributes can be processed, including those attributes that can be processed in parallel.
Figure 24 displays an example execution tree for a Time dimension. The solid arrows represent the attribute relationships in the dimension. The dashed arrows represent the implicit relationship of each attribute to the All attribute. Note that the dimension has been configured using cascading attribute relationships which is a best practice for all dimension designs.
Figure 24 Execution tree example
In this example, the All attribute proceeds first, given that it has no dependencies to another attribute, followed by the Fiscal Year and Calendar Year attributes, which can be processed in parallel. The other attributes proceed according to the dependencies in the execution tree with the primary key attribute always being processed last since it always has at least one attribute relationship, except when it is the only attribute in the dimension.
The time taken to process an attribute is generally dependent on 1) the number of members and 2) the number of attribute relationships. While you cannot control the number of members for a given attribute, you can improve processing performance by using cascading attribute relationships. This is especially critical for the key attribute since it has the most members and all other jobs (hierarchy, decoding, bitmap indexes) are waiting for it to complete. For more information about the importance of using cascading attribute relationships, see Identifying attribute relationships.
Build decoding stores
Decoding stores are used extensively by the Storage Engine. During querying, they are used to retrieve data from the dimension. During processing, they are used to build the dimension’s bitmap indexes.
Build hierarchy stores
A hierarchy store is a persistent representation of the tree structure. For each natural hierarchy in the dimension, a job is instantiated to create the hierarchy stores. For more information on hierarchy stores, see Data retrieval: dimensions.
Build bitmap indexes
To efficiently locate attribute data in the relationship store at querying time, the Storage Engine creates bitmap indexes at processing time. For attributes with a very large number of DataIDs, the bitmap indexes can take some time to process. In most scenarios, the bitmap indexes provide significant querying benefits; however, when you have high cardinality attributes, the querying benefit that the bitmap index provides may not outweigh the processing cost of creating the bitmap index. For more information on this design scenario, see Reducing attribute overhead.
Dimension-processing commands
When you need to perform a process operation on a dimension, you issue dimension processing commands. Each processing command creates one or more jobs to perform the necessary operations.
From a performance perspective, the following dimension processing commands are the most important:
- A ProcessFull command discards all storage contents of the dimension and rebuilds them. Behind the scenes, ProcessFull executes all dimension processing jobs and performs an implicit ProcessClear to discard the storage contents of all dependent partitions. This means that whenever you perform a ProcessFull of a dimension, you need to perform a ProcessFull on dependent partitions to bring the cube back online.
- ProcessData discards all storage contents of the dimension and rebuilds only the attribute and hierarchy stores. ProcessData is a component of the ProcessFull operation. ProcessData also clears partitions.
- ProcessIndexes requires that a dimension already has attribute and hierarchy stores built. ProcessIndexes preserves the data in these stores and then rebuilds the bitmap indexes. ProcessIndexes is a component of the ProcessFull operation.
- ProcessUpdate does not discard the dimension storage contents, unlike ProcessFull. Rather, it applies updates intelligently in order to preserve dependent partitions. More specifically, ProcessUpdate sends SQL queries to read the entire dimension table and then applies changes to the dimension stores. A ProcessUpdate can handle inserts, updates, and deletions depending on the type of attribute relationships (rigid vs. flexible) in the dimension. Note that ProcessUpdate will drop invalid aggregations and indexes, requiring you to take action to rebuild the aggregations in order to maintain query performance. For more information on applying ProcessUpdate, see Evaluating rigid vs. flexible aggregations.
- ProcessAdd optimizes ProcessUpdate in scenarios where you only need to insert new members. ProcessAdd does not delete or update existing members. The performance benefit of ProcessAdd is that you can use a different source table or data source view named query that restrict the rows of the source dimension table to only return the new rows. This eliminates the need to read all of the source data. In addition, ProcessAdd also retains flexible aggregations.
For a more comprehensive list of processing commands, see the Analysis Services 2005 Processing Architecture white paper located on the Microsoft Developer Network (MSDN).
Partition-processing jobs
During partition processing, source data is extracted and stored on disk using the series of jobs displayed in Figure 25.
Figure 25 Partition processing jobs
Process fact data
Fact data is processed using three concurrent threads that perform the following tasks:
- Send SQL statements to extract data from data sources.
- Look up dimension keys in dimension stores and populate the processing buffer.
- When the processing buffer is full, write out the buffer to disk.
During the processing of fact data, a potential bottleneck may be the source SQL statement. For techniques to optimize the source SQL statement, see Optimizing the source query.
Build aggregations and bitmap indexes
Aggregations are built in memory during processing. While too few aggregations may have little impact on query performance, excessive aggregations can increase processing time without much added value on query performance. As a result, care must be taken to ensure that your aggregation design supports your required processing window. For more information on deciding which aggregations to build, see Adopting an aggregation design strategy.
If aggregations do not fit in memory, chunks are written to temp files and merged at the end of the process. Bitmap indexes are built on the fact and aggregation data and written to disk on a segment by segment basis.
Partition-processing commands
When you need to perform a process operation on a partition, you issue partition processing commands. Each processing command creates one or more jobs to perform the necessary operations.
From a performance perspective, the following partition processing commands are the most important:
- ProcessFull discards the storage contents of the partition and rebuilds them. Behind the scenes, a ProcessFull executes ProcessData and ProcessIndexes jobs.
- ProcessData discards the storage contents of the object and rebuilds only the fact data.
- ProcessIndexes requires that a partition already has its data built. ProcessIndexes preserves the data and any existing aggregations and bitmap indexes and creates any missing aggregations or bitmap indexes.
- ProcessAdd internally creates a temporary partition, processes it with the target fact data, and then merges it with the existing partition. Note that ProcessAdd is the name of the XMLA command. This command is exposed in Business Intelligence Development Studio and SQL Server Management Studio as ProcessIncremental.
For a more comprehensive list of processing commands, see Analysis Services 2005 Processing Architecture white paper on MSDN.
Executing processing jobs
To manage dependencies among jobs, the Analysis Services server organizes jobs into a processing schedule. Dimensions, for example, must always be processed first, given the inherent dependency of partitions on dimensions.
Jobs without dependencies can be executed in parallel as long as there are available system resources to carry out the jobs. For example, multiple dimensions can be processed in parallel, multiple measure groups can be processed in parallel, and multiple partitions within a measure group can be processed in parallel. Analysis Services performs as many operations as it can in parallel based on available resources and the values of the following three properties: the CoordinatorExecutionMode server property, and the MaxParallel processing command, and the Threadpool\Process\MaxThreads server property. For more information on these properties, see Maximize parallelism during processing in this white paper.
In addition, before executing a processing job, Analysis Services verifies the available memory. Each job requests a specific amount of memory from the Analysis Services memory governor. If there is not enough memory available to fulfill the memory request, the memory governor can block the job. This behavior can be especially relevant in memory-constrained environments when you issue a processing request that performs multiple intensive operations such as a ProcessFull on a large partition that contains a complex aggregation design. For more information on optimizing this scenario, see Tuning memory for partition processing.
In situations where you are performing querying and processing operations at the same time, a long-running query can block a processing operation and cause it to fail. When you encounter this, the processing operation unexpectedly cancels, returning a generic error message. During the execution of a query, Analysis Services takes a read database commit lock. During processing, Analysis Services requires a write database commit lock. The ForceCommitTimeout server property identifies the amount of time a process operation waits before killing any blocking read locks. Once the timeout threshold has been reached, all transactions holding the lock will fail. The default value for this property is 30,000 milliseconds (30 seconds). Note that this property can be modified in the msmdsrv.ini configuration file; however, it is generally not recommended that you modify this setting. Rather, it is simply important to understand the impact of long-running queries on concurrent processing to help you troubleshoot any unexpected processing failures.
Refreshing dimensions efficiently
As stated previously, the performance goal of dimension processing is to refresh dimension data in an efficient manner that does not negatively impact the query performance of dependent partitions. To accomplish this, you can perform the following techniques: optimize SQL source queries, reduce attribute overhead, and prepare each dimension attribute to efficiently handle inserts, updates, deletes, as necessary.
Optimizing the source query
During processing, you can optimize the extraction of dimension source data using the following techniques:
Use OLE DB Providers over .NET Data Providers
Since the Analysis Services runtime is written in native code, OLE DB Providers offer performance benefits over .NET Data Providers. When you use .NET Data Providers, data has to be marshaled between the .NET managed memory space and the native memory space. Since OLE DB Providers are already in native code, they provide significant performance benefits over.NET Data Providers and should be used whenever possible.
Use attribute relationships to optimize attribute processing across multiple data sources
When a dimension comes from multiple data sources, using cascading attribute relationships allows the system to segment attributes during processing according to data source. If an attribute’s key, name, and attribute relationships come from the same database, the system can optimize the SQL query for that attribute by querying only one database. Without cascading attribute relationships, the SQL Server OPENROWSET function is used to merge the data streams. The OPENROWSET function provides a mechanism to access data from multiple data sources. For each attribute, a separate OPENROWSET-derived table is used. In this situation, the processing for the key attribute is extremely slow since it must access multiple OPENROWSET derived tables.
Tune the Processing Group property
When a dimension is processed, the default behavior is to issue a separate SQL statement that retrieves a distinct set of members for each attribute. This behavior is controlled by the Processing Group property, which is automatically set to ByAttribute. In most scenarios, ByAttribute provides the best processing performance; however, there are a few niche scenarios where it can be useful to change this property to ByTable. In ByTable, Analysis Services issues a single SQL statement on a per-table basis to extract a distinct set of all dimension attributes. This is potentially beneficial in scenarios where you need to process many high cardinality attributes and you are waiting a long time for the select distinct to complete for each attribute. Note that whenever ByTable is used, the server changes its processing behavior to use a multi-pass algorithm. A similar algorithm is also used for very large dimensions when the hash tables of all related attributes do not fit into memory. The algorithm can be very expensive in some scenarios because Analysis Services must read and store to disk all the data from the table in multiple data stores, and then iterate over it for each attribute. Therefore, whatever performance benefit you gain for quickly evaluating the SQL statement could be counteracted by the other processing steps. So while ByTable has the potential to be faster, it is only appropriate in scenarios where you believe that issuing one SQL statement performs significantly better than issuing multiple smaller SQL statements. In addition, note that if you use ByTable, you will see duplicate member messages when processing the dimension if you have configured the KeyDuplicate property to ReportAndContinue or ReportAndStop. In this scenario, these duplicate error messages are false positives resulting from the fact that the SQL statement is no longer returning a distinct set of members for each attribute. To better understand how these duplicates occur, consider the following example. You have a customer dimension table that has three attributes: customer key, customer name, and gender. If you set the Processing Group property to ByTable, one SQL statement is used to extract all three attributes. Given the granularly of the SQL statement, in this scenario, you will see duplicate error messages for the gender attribute if you have set the KeyDuplicate property to raise an error. Again, these error messages are likely false positives; however, you should still examine the messages to ensure that there are no unexpected duplicate values in your data.
Reducing attribute overhead
Every attribute that you include in a dimension impacts the cube size, the dimension size, the aggregation design, and processing performance. Whenever you identify an attribute that will not be used by end users, delete the attribute entirely from your dimension. Once you have removed extraneous attributes, you can apply a series of techniques to optimize the processing of remaining attributes.
Use the KeyColumns and NameColumn properties effectively
When you add a new attribute to a dimension, two properties are used to define the attribute. The KeyColumns property specifies one or more source fields that uniquely identify each instance of the attribute and the NameColumn property specifies the source field that will be displayed to end users. If you do not specify a value for the NameColumn property, it is automatically set to the value of the KeyColumns property.
Analysis Services provides the ability to source the KeyColumns and NameColumn properties from different source columns. This is useful when you have a single entity like a product that is identified by two different attributes: a surrogate key and a descriptive product name. When users want to slice data by products, they may find that the surrogate key lacks business relevance and will choose to use the product name instead.
From a processing perspective, it is a best practice to assign a numeric source field to the KeyColumns property rather than a string property. Not only can this reduce processing time, in some scenarios it can also reduce the size of the dimension. This is especially true for attributes that have a large number of members, i.e., greater than 1 million members.
Rather than using a separate attribute to store a descriptive name, you can use the NameColumn property to display a descriptive field to end users. In the product example, this means you can assign the surrogate key to the KeyColumns property and use the product name to the NameColumn property. This eliminates the need for the extraneous name attribute, making your design more efficient to query and process.
Remove bitmap indexes
During processing of the primary key attribute, bitmap indexes are created for every related attribute. Building the bitmap indexes for the primary key can take time if it has one or more related attributes with high cardinality. At querying time, the bitmap indexes for these attributes are not useful in speeding up retrieval since the Storage Engine still must sift through a large number of distinct values.
For example, the primary key of the customer dimension uniquely identifies each customer by account number; however, users also want to slice and dice data by the customer’s social security number. Each customer account number has a one-to-one relationship with a customer social security number. To avoid spending time building unnecessary bitmap indexes for the social security number attribute, it is possible to disable its bitmap indexes by setting the AttributeHierarchyOptimizedState property to Not Optimized.
Turn off the attribute hierarchy and use member properties
As an alternative to attribute hierarchies, member properties provide a different mechanism to expose dimension information. For a given attribute, member properties are automatically created for every attribute relationship. For the primary key attribute, this means that every attribute that is directly related to the primary key is available as a member property of the primary key attribute.
If you only want to access an attribute as member property, once you verify that the correct relationship is in place, you can disable the attribute’s hierarchy by setting the AttributeHierarchyEnabled property to False. From a processing perspective, disabling the attribute hierarchy can improve performance and decrease cube size because the attribute will no longer be indexed or aggregated. This can be especially useful for high cardinality attributes that have a one-to-one relationship with the primary key. High cardinality attributes such as phone numbers and addresses typically do not require slice-and-dice analysis. By disabling the hierarchies for these attributes and accessing them via member properties, you can save processing time and reduce cube size.
Deciding whether to disable the attribute’s hierarchy requires that you consider both the querying and processing impacts of using member properties. Member properties cannot be placed on a query axis in the same manner as attribute hierarchies and user hierarchies. To query a member property, you must query the properties of the attribute that contains the member property. For example, if you require the work phone number for a customer, you must query the properties of customer. As a convenience, most front-end tools easily display member properties in their user interfaces.
In general, querying member properties can be slower than querying attribute hierarchies because member properties are not indexed and do not participate in aggregations. The actual impact to query performance depends on how you are going to use the attribute. If your users want to slice and dice data by both account number and account description, from a querying perspective you may be better off having the attribute hierarchies in place and removing the bitmap indexes if processing performance is an issue. However, if you are simply displaying the work phone number on a one-off basis for a particular customer and you are spending large amounts of time in processing, disabling the attribute hierarchy and using a member property provides a good alternative.
Optimizing dimension inserts, updates, and deletes
Dimension data refreshes can be generally handled via three processing operations:
- ProcessFull—Erases and rebuilds the dimension data and structure.
- ProcessUpdate—Implements inserts, updates, and deletes based on the types of attribute relationships in the dimension. Information on the different types of attribute relationships is included later in this section.
- ProcessAdd—Provides an optimized version of ProcessUpdate to only handle data insertions.
As you plan a dimension refresh, in addition to selecting a processing operation, you must also assess how each attribute and attribute relationship is expected to change over time. More specifically, every attribute relationship has a Type property that determines how the attribute relationship should respond to data changes. The Type property can either be set to flexible or rigid where flexible is the default setting for every attribute relationship.
Use flexible relationships
Flexible relationships permit you to make a variety of data changes without requiring you to use a ProcessFull to completely rebuild the dimension every time you make a change. Remember that as soon as you implement a ProcessFull on a dimension, the cube is taken offline and you must perform a ProcessFull on every dependent partition in order to restore the ability to query the cube.
For attributes with flexible relationships, inserts, updates, and deletions can be handled by using the ProcessUpdate command. The ProcessUpdate command allows you to keep the cube “online” while the data changes are made. By default, every attribute relationship is set to flexible, although it may not always be the best choice in every design scenario. Using flexible relationships is appropriate whenever you expect an attribute to have frequent data changes and you do not want to experience the impacts of performing a ProcessFull on the dimension and cube. For example, if you expect products to frequently change from one category to another, you may decide to keep the flexible relationship between product and category so that you only need to perform a ProcessUpdate to implement the data change.
The tradeoff to using flexible relationships is their impact on data aggregations. For more information on flexible aggregations, see Evaluating rigid vs. flexible aggregations.
Note that in some processing scenarios, flexible relationships can “hide” invalid data changes in your dimension. As stated in the Identifying attribute relationships section, whenever you use attribute relationships, you must verify that each attribute’s KeyColumns property uniquely identifies each attribute member. If the KeyColumns property does not uniquely identify each member, duplicates encountered during processing are ignored by default, resulting in incorrect data rollups. More specifically, when Analysis Services encounters a duplicate member, it picks an arbitrary member depending on the processing technique that you have selected. If you perform a ProcessFull, it selects the first member it finds. If you perform a ProcessUpdate, it selects the last member it finds. To avoid this scenario, follow the best practice of always assigning the KeyColumns property with a column or combination of columns that unique identifies the attribute. If you follow this practice, you will not encounter this problem. In addition, it is a good practice to change the default error configuration to no longer ignore duplicates. To accomplish this, set the KeyDuplicate property from IgnoreError to ReportAndContinue or ReportAndStop. With this change, you can be alerted of any situation where duplicates are detected. However, this option may give you false positives in some cases (e.g., ByTable processing). For more information, see Optimizing the source query.
Use rigid relationships
For attributes with rigid relationships, inserts can be handled by using a ProcessAdd or ProcessUpdate to the dimension; however, updates and deletions require a ProcessFull of the dimension and consequently require a ProcessFull of the dependent partitions. As such, rigid relationships are most appropriate for attributes that have zero or infrequent updates or deletions. For example, in a time dimension you may assign a rigid relationship between month and quarter since the months that belong to a given quarter do not change.
If you want to assign a relationship as rigid (remember that the relationships are flexible by default), you must ensure that the source data supports the rigid relationship, i.e., no changes can be detected when you perform a ProcessAdd or ProcessUpdate. If Analysis Services detects a change, the dimension process fails and you must perform a ProcessFull. In addition, if you use rigid relationships, duplicate members are never tolerated for a given attribute, unlike in flexible relationships where an arbitrary member is selected during processing. Therefore, you must also ensure that your KeyColumns property is correctly configured.
Similar to flexible relationships, when you define rigid relationships, you must also understand their impact on data aggregations. For more information on rigid aggregations, see Evaluating rigid vs. flexible aggregations.
Refreshing partitions efficiently
The performance goal of partition processing is to refresh fact data and aggregations in an efficient manner that satisfies your overall data refresh requirements. To help you refresh your partitions, the following techniques are discussed in this topic: optimizing SQL source queries, using multiple partitions, effectively handling data inserts, updates, and deletes, and evaluating the usage of rigid vs. flexible aggregations.
Optimizing the source query
To enhance partition processing, there are two general best practices that you can apply for optimizing the source query that extracts fact data from the source database.
Use OLE DB Providers over .NET Data Providers
This is the same recommendation provided for the dimension processing. Since the Analysis Services runtime is written in native code, OLE DB Providers offer performance benefits over.NET Data Providers. When you use .NET Data Providers, data has to be marshaled between the .NET managed memory space and the native memory space. Since OLE DB Providers are already in native code, they provide significant performance benefits over the .NET Data Providers.
Use query bindings for partitions
A partition can be bound to either a source table or a source query. When you bind to a source query, as long as you return the correct number of columns expected in the partition, you can create a wide range of SQL statements to extract source data. Using source query bindings provides greater flexibility than using a named query in the data source view. For example, using query binding, you can point to a different data source using four-part naming and perform additional joins as necessary.
Using partitions to enhance processing performance
In the same manner that using multiple partitions can reduce the amount of data that needs to be scanned during data retrieval (as discussed in How partitions are used in querying), using multiple partitions can enhance processing performance by providing you with the ability to process smaller data components of a measure group in parallel.
Being able to process multiple partitions in parallel is useful in a variety of scenarios; however, there are a few guidelines that you must follow. When you initially create a cube, you must perform a ProcessFull on all measure groups in that cube.
If you process partitions from different client sessions, keep in mind that whenever you process a measure group that has no processed partitions, Analysis Services must initialize the cube structure for that measure group. To do this, it takes an exclusive lock that prevents parallel processing of partitions. If this is the case, you should ensure that you have at least one processed partition per measure group before you begin parallel operations. If you do not have a processed partition, you can perform a ProcessStructure on the cube to build its initial structure and then proceed to process measure group partitions in parallel. In the majority of scenarios, you will not encounter this limitation if you process partitions in the same client session and use the MaxParallel XMLA element to control the level of parallelism. For more information on using MaxParallel, see Maximize parallelism during processing.
After initially loading your cube, multiple partitions are useful when you need to perform targeted data refreshes. Consider the following example. If you have a sales cube with a single partition, every time that you add new data such as a new day’s worth of sales, you must not only refresh the fact data, but you must also refresh the aggregations to reflect the new data totals. This can be costly and time consuming depending on how much data you have in the partition, the aggregation design for the cube, and the type of processing that you perform.
With multiple partitions, you can isolate your data refresh operations to specific partitions. For example, when you need to insert new fact data, an effective technique involves creating a new partition to contain the new data and then performing a ProcessFull on the new partition. Using this technique, you can avoid impacting the other existing partitions. Note that it is possible to use XMLA scripts to automate the creation of a new partition during the refresh of your relational data warehouse.
Optimizing data inserts, updates, and deletes
This section provides guidance on how to efficiently refresh partition data to handle inserts, updates, and deletes.
Inserts
If you have a browseable cube and you need to add new data to an existing measure group partition, you can apply one of the following techniques:
- ProcessFull—Perform a ProcessFull for the existing partition. During the ProcessFull operation, the cube remains available for browsing with the existing data while a separate set of data files are created to contain the new data. When the processing is complete, the new partition data is available for browsing. Note that a ProcessFull is technically not necessary given that you are only doing inserts. To optimize processing for insert operations, you can use a ProcessAdd.
- ProcessAdd—Use this operation to append data to the existing partition files. If you frequently perform a ProcessAdd, it is advised that you periodically perform a ProcessFull in order to rebuild and re-compress the partition data files. ProcessAdd internally creates a temp partition and merges it. This results in data fragmentation over time and the need to periodically perform a ProcessFull.
If your measure group contains multiple partitions, as described in the previous section, a more effective approach is to create a new partition that contains the new data and then perform a ProcessFull on that partition. This technique allows you to add new data without impacting the existing partitions. When the new partition has completed processing, it is available for querying.
Updates
When you need to perform data updates, you can perform a ProcessFull. Of course it is useful if you can target the updates to a specific partition so you only have to process a single partition. Rather than directly updating fact data, a better practice is to use a “journaling” mechanism to implement data changes. In this scenario, you turn an update into an insertion that corrects that existing data. With this approach, you can simply continue to add new data to the partition by using a ProcessAdd. You can also have an audit trail of the changes that you have made.
Deletes
For deletions, multiple partitions provide a great mechanism for you to roll out expired data. Consider the following example. You currently have 13 months of data in a measure group, one month per partition. You want to roll out the oldest month from the cube. To do this, you can simply delete the partition without affecting any of the other partitions. If there are any old dimension members that only appeared in the expired month, you can remove these using a ProcessUpdate of the dimension (as long it contains flexible relationships). In order to delete members from the key/granularity attribute of a dimension, you must set the dimension’s UnknownMember property to Hidden or Visible. This is because the server does not know if there is a fact record assigned to the deleted member. With this property set appropriately, it will associate it to the unknown member at query time.
Evaluating rigid vs. flexible aggregations
Flexible and rigid attribute relationships not only impact how you process dimensions, but they also impact how aggregations are refreshed in a partition. Aggregations can either be categorized as rigid or flexible depending on the relationships of the attributes participating in the aggregation. For more information on the impact of rigid and flexible relationships, see Optimizing dimension inserts, updates, and deletes.
Rigid aggregations
An aggregation is rigid when all of the attributes participating in the aggregation have rigid direct or indirect relationships to the granularity attribute of a measure group. For all attributes in the aggregation, a check is performed to verify that all relationships are rigid. If any are flexible, the aggregation is flexible.
Flexible aggregations
An aggregation is flexible when one or more of the attributes participating in the aggregation have flexible direct or indirect relationships to the key attribute.
If you perform a ProcessUpdate on a dimension participating in flexible aggregations, whenever deletions or updates are detected for a given attribute, the aggregations for that attribute as well as any related attributes in the attribute chain are automatically dropped. The aggregations are not automatically recreated unless you perform one of the following tasks:
- Perform a ProcessFull on the cube, measure group, or partition. This is the standard ProcessFull operation that drops and re-creates the fact data and all aggregations in the partition.
- Perform a ProcessIndexes on the
cube, measure group, or partition.
By performing ProcessIndexes, you will only build whatever aggregations or indexes that need to be re-built.
- Configure Lazy Processing for the cube, measure group, or partition. If you configure Lazy Processing, the dropped aggregations are recalculated as a background task. While the flexible aggregations are being recalculated, users can continue to query the cube (without the benefit of the flexible aggregations). While the flexible aggregations are being recalculated, queries that would benefit from the flexible aggregations run slower because Analysis Services resolves these queries by scanning the fact data and then summarizing the data at query time. As the flexible aggregations are recalculated, they become available incrementally on a partition-by-partition basis. For a given cube, Lazy Processing is not enabled by default. You can configure it for a cube, measure group, or partition by changing the ProcessingMode property from Regular to LazyAggregations. To manage Lazy Processing, there are a series of server properties such as the LazyProcessing \ MaxObjectsInParallel setting, which controls the number of objects that can be lazy processed at a given time. By default it is set to 2. By increasing this number, you increase the number of objects processed in parallel; however, this also impacts query performance and should therefore be handled with care.
- Process affected objects. When you perform a ProcessUpdate on a dimension, you can choose whether or not to Process Affected Objects. If you select this option, you can trigger the update of dependent partitions within the same operation.
Note that if you do not follow one of the above techniques, and you perform a ProcessUpdate of a dimension that results in a deletion or update, the flexible aggregations for that attribute and all related attributes in the attribute chain are automatically deleted and not re-created, resulting in poor query performance. This is especially important to note because by default every aggregation is flexible since every attribute relationship type is set to Flexible.
As a result, great care must be taken to ensure that your refresh strategy configures the appropriate attribute relationships for your data changes and effectively rebuilds any flexible aggregations on an ongoing basis.
Optimizing Special Design Scenarios
Throughout this whitepaper, specific techniques and best practices are identified for improving the processing and query performance of Analysis Services OLAP databases. In addition to these techniques, there are specific design scenarios that require special performance tuning practices. Following is an overview of the design scenarios that are addressed in this section:
Special aggregate functions – Special aggregate functions allow you to implement distinct count and semiadditive data summarizations. Given the unique nature of these aggregate functions, special performance tuning techniques are required to ensure that they are implemented in the most efficient manner.
Parent-child hierarchies – Parent-child hierarchies have a different aggregation scheme than attribute and user hierarchies, requiring that you consider their impact on query performance in large-scale dimensions.
Complex Dimension Relationships – Complex dimension relationships include many-to-many relationships and reference relationships. While these relationships allow you to handle a variety of schema designs, complex dimension relationships also require you to assess how the schema complexity is going to impact processing and/or query performance.
Near real-time data refreshes – In some design scenarios, “near real-time” data refreshes are a necessary requirement. Whenever you implement a “near real-time” solution requiring low levels of data latency, you must consider how you are going to balance the required latency with querying and processing performance.
Special aggregate functions
Aggregate functions are the most common mechanism to summarize data. Aggregate functions are uniformly applied across all attributes in a dimension and can be used for additive, semiadditive, and nonadditive measures. Aggregate functions can be categorized into two general groups: traditional aggregate functions and special aggregate functions.
- Traditional aggregate functions consist of a group of functions that follow the general performance tuning recommendations described in other sections of this white paper. Traditional aggregate functions include Sum, Count, Min, and Max.
- Special aggregate functions require unique performance-tuning techniques. They include DistinctCount and a collection of semiadditive aggregate functions. The semiadditive functions include: FirstChild, LastChild, FirstNonEmpty, LastNonEmpty, ByAccount, and AverageOfChildren.
Optimizing distinct count
DistinctCount is a nonadditive aggregate function that counts the unique instances of an entity in a measure group. While distinct count is a very powerful analytical tool, it can have significant impact on processing and querying performance because of its explosive impact on aggregation size. When you use a distinct count aggregate function, it increases the size of an aggregation by the number of unique instances that are distinctly counted.
To better understand how distinct count impacts aggregation size, consider the following example. You have a measure group partition with an aggregation that summarizes sales amount by product category and year. You have ten product categories with sales for ten years, producing a total of 100 values in the aggregation. When you add a distinct count of customers to the measure group, the aggregation for the partition changes to include the customer key of each customer who has sales for a specific product category and year. If there are 1,000 customers, the number of potential values in the aggregation increases from 100 to 100,000 values, given that every customer has sales for every product category in every year. (Note that the actual number of values would be less than 100,000 due to natural data sparsity. At any rate, the value is likely to be a number much greater than 100.) While this additional level of detail is necessary to efficiently calculate the distinct count of customers, it introduces significant performance overhead when users request summaries of sales amount by product category and year.
With the explosive impact of distinct counts on aggregations, it is a best practice to separate each distinct count measure into its own measure group with the same dimensionality as the initial measure group. Using this technique, you can isolate the distinct count aggregations and maintain a separate aggregation design for non-distinct count measures.
Note that when you use Business Intelligence Development Studio to create a new measure, if you specify the distinct count aggregate function at the time of the measure creation, the Analysis Services Cube Designer automatically creates a separate measure group for the distinct count measure. However, if you change the aggregate function for an existing measure to distinct count, you must manually reorganize the distinct count measure into its own measure group.
When you are distinctly counting large amount of data, you may find that the aggregations for the distinct count measure group are not providing significant value, since the fact data must typically be queried to calculate the distinct counts. For a distinct count measure group, you should consider partitioning the measure group using data ranges of the distinct count field. For example, if you are performing a distinct count of customers using customer ID, consider partitioning the distinct count measure group by customer ID ranges. In this scenario, partition 1 may contain customer IDs between 1 and 1000. Partition 2 may contain customer IDs between 1001 and 2000. Partition 3 may contain customer IDs between 2001 and 3000, etc. This partitioning scheme improves query parallelism since the server does not have to coordinate data across partitions to satisfy the query.
For larger partitions, to further enhance performance, it may be advantageous to consider an enhanced partitioning strategy where you partition the distinct count measure group by both the distinct count field as well as your normal partitioning attribute, i.e., year, month, etc. As you consider your partitioning strategy for distinct count, keep in mind that when you partition by multiple dimensions, you may quickly find out that you have too many partitions. With too many partitions, in some scenarios you can actually negatively impact query performance if Analysis Services needs to scan and piece together data from multiple partitions at query time. For example, if you partition by sales territory and day, when users want the distinct count of customers by sales territory by year, all of the daily partitions in a specific year must be accessed and assimilated to return the correct value. While this will naturally happen in parallel, it can potentially result in additional querying overhead. To help you determine the size of and scope of your partitions, see the general sizing guidance in the Designing partitions section.
From a processing perspective, whenever you process a partition that contains a distinct count, you will notice an increase in processing time over your other partitions of the same dimensionality. It takes longer to process because of the increased size of the fact data and aggregations. The larger the amount of data that requires processing, the longer it takes to process, and the greater potential that Analysis Services may encounter memory constraints and will need to use temporary files to complete the processing of the aggregation. You must therefore assess whether your aggregation design is providing you with the most beneficial aggregations. To help you determine this, you can use the Usage-Based Optimization Wizard to customize your aggregation design to benefit your query workload. If you want further control over the aggregation design, you may want to consider creating custom aggregations by using the Aggregation Utility described in Appendix C. In addition, you must ensure that the system has enough resources to process the aggregations. For more information on this topic, see Tuning Server Resources.
The other reason that processing time may be slower is due to the fact that an ORDER BY clause is automatically added to the partition’s source SQL statement. To optimize the performance of the ORDER BY in the source SQL statement, you can place a clustered index on the source table column that is being distinctly counted. Keep in mind that this is only applicable in the scenario where you have one distinct count per fact table. If you have multiple distinct counts off of a single fact table, you need to evaluate which distinct count will benefit most from the clustered index. For example, you may choose to apply the clustered index to the most granular distinct count field.
Optimizing semiadditive measures
Semiadditive aggregate functions include ByAccount, AverageOfChildren, FirstChild, LastChild, FirstNonEmpty, and LastNonEmpty. To return the correct data values, semiadditive functions must always retrieve data that includes the granularity attribute of the time dimension. Since semiadditive functions require more detailed data and additional server resources, to enhance performance, semiadditive measures should not be used in a cube that contains ROLAP dimensions or linked dimensions. ROLAP and linked dimensions also require additional querying overhead and server resources. Therefore, the combination of semiadditive measures with ROLAP and/or linked dimensions can result in poor query performance and should be avoided where possible.
Aggregations containing the granularity attribute of the time dimension are extremely beneficial for efficiently satisfying query requests for semiadditive measures. Aggregations containing other time attributes are never used to fulfill the request of a semiadditive measure.
To further explain how aggregations are used to satisfy query requests for semiadditive measures, consider the following example. In an inventory application, when you request a product’s quantity on hand for a given month, the inventory balance for the month is actually the inventory balance for the final day in that month. To return this value, the Storage Engine must access partition data at the day level. If there is an appropriate aggregation that includes the day attribute, the Storage Engine will attempt to use that aggregation to satisfy the query. Otherwise the query must be satisfied by the partition fact data. Aggregations at the month and year cannot be used.
Note that if your cube contains only semiadditive measures, you will never receive performance benefits from aggregations created for nongranularity time attributes. In this scenario, you can influence the aggregation designer to create aggregations only for the granularity attribute of the time dimension (and the All attribute) by setting the Aggregation Usage to None for the nongranularity attributes of the time dimension. Keep in mind that the Aggregation Usage setting automatically applies to all measure groups in the cube, so if the cube contains only semiadditive measures, such as in inventory applications, the Aggregation Usage setting can be uniformly applied across all measure groups.
If you have a mixed combination of semiadditive measures and other measures across measure groups, you can adopt the following technique to apply unique aggregation usage settings to specific measure groups. First, adjust the Aggregation Usage settings for a dimension so that they fit the needs of a particular measure group and then design aggregations for that measure group only. Next, change the Aggregation Usage settings to fit the needs of the next measure group and then design aggregations for that measure group only. Using this approach, you can design aggregations measure group-by-measure group, and even partition-by-partition if desired.
In addition to this technique, you can also optimize the aggregation design using the Usage-Based Optimization Wizard to create only those aggregations that provide the most value for your query patterns. If you want further control over the aggregation design, you may want to consider creating custom aggregations using the Aggregation Utility described in Appendix C.
Parent-child hierarchies
Parent-child hierarchies are hierarchies with a variable number of levels, as determined by a recursive relationship between a child attribute and a parent attribute. Parent-child hierarchies are typically used to represent a financial chart of accounts or an organizational chart. In parent-child hierarchies, aggregations are created only for the key attribute and the top attribute, i.e., the All attribute unless it is disabled. As such, refrain from using parent-child hierarchies that contain large numbers of members at intermediate levels of the hierarchy. Additionally, you should limit the number of parent-child hierarchies in your cube.
If you are in a design scenario with a large parent-child hierarchy (greater than 250,000 members), you may want to consider altering the source schema to re-organize part or all of the hierarchy into a user hierarchy with a fixed number of levels. Once the data has been reorganized into the user hierarchy, you can use the Hide Member If property of each level to hide the redundant or missing members.
Complex dimension relationships
The flexibility of Analysis Services enables you to build cubes from a variety of source schemas. For more complicated schemas, Analysis Services supplies many-to-many relationships and reference relationships to help you model complex associations between your dimension tables and fact tables. While these relationships provide a great deal of flexibility, to use them effectively, you must evaluate their impact on processing and query performance.
Many-to-many relationships
In Analysis Services, many-to-many relationships allow you to easily model complex source schemas. To use many-to-many relationships effectively, you must be familiar with the business scenarios where these relationships are relevant.
Background information on many-to-many relationships
In typical design scenarios, fact tables are joined to dimension tables via many-to-one relationships. More specifically, each fact table record can join to only one dimension table record and each dimension table record can join to multiple fact table records. Using this many-to-one design, you can easily submit queries that aggregate data by any dimension attribute.
Many-to-many design scenarios occur when a fact table record can potentially join to multiple dimension table records. When this situation occurs, it is more difficult to correctly query and aggregate data since the dimension contains multiple instances of a dimension entity and the fact table cannot distinguish among the instances.
To better understand how many-to-many relationships impact data analysis, consider the following example. You have a reseller dimension where the primary key is the individual reseller. To enhance your analysis, you want to add the consumer specialty attribute to the reseller dimension. Upon examining data, you notice that each reseller can have multiple consumer specialties. For example, A Bike Store has two consumer specialties: Bike Enthusiast and Professional Bike. Your sales fact table, however, only tracks sales by reseller, not by reseller and consumer specialty. In other words, the sales data does not distinguish between A Bike Shop with a Bike Enthusiast specialty and A Bike Shop with a Professional Bike specialty. In the absence of some kind of weighting factor to allocate the sales by consumer specialty, it is a common practice to repeat the sales values for each reseller/consumer specialty combination. This approach works just fine when you are viewing data by consumer specialty and reseller; however, it poses a challenge when you want to examine data totals as well as when you want to analyze data by other attributes.
Continuing with the reseller example, if you add the new consumer specialty attribute to the reseller dimension table, whenever the fact table is joined to the dimension table via the individual reseller, the number of records in the result set is multiplied by the number of combinations of reseller and consumer specialty. When the data from this result set is aggregated, it will be inflated. In the example of A Bike Shop, sales data from the fact table will be double-counted regardless of which reseller attribute you group by. This double-counting occurs because A Bike Shop appears twice in the table with two consumer specialties. This data repeating is okay when you are viewing the breakdown of sales by consumer specialty and reseller; however, any sales totals or queries that do not include consumer specialty will be incorrectly inflated.
To avoid incorrectly inflating your data, many-to-many relationships are typically modeled in the source schema using multiple tables as displayed in Figure 26.
- A main dimension table contains a unique list of primary key values and other attributes that have a one-to-one relationship to the primary key. To enable the dimension table to maintain a one-to-many relationship with the fact table, the multivalued attribute is not included in this dimension. In the reseller example displayed in Figure 26, Dim Reseller is the main dimension table, storing one record per reseller. The multivalued attribute, consumer specialty, is not included in this table.
- A second dimension table contains a unique list of values for the multivalued attribute. In the reseller example, the Dim Consumer Specialty dimension table stores the distinct list of consumer specialties.
- An intermediate fact table maps the relationship between the two dimension tables. In the reseller example, Fact Reseller Specialty is an intermediate fact table that tracks the consumer specialties of each reseller.
- The data fact table stores a foreign key reference to the primary key of the main dimension table. In the reseller example, the Fact Sales table stores the numeric sales data and has a many-to-one relationship with Dim Reseller via the Reseller Key.
Figure 26 Many-to-many relationship
Using this design, you have maintained the many-to-one relationship between Fact Sales and Dim Reseller. This means that if you ignore Dim Consumer Specialty and Fact Reseller Specialty, you can easily query and aggregate data from Fact Sales by any attribute in the Dim Reseller table without incorrectly inflating values. However this design does not fully solve the problem, since it is still tricky to analyze sales by an attribute in Dim Consumer Specialty. Remember that you still do not have sales broken down by consumer specialty; i.e., sales is only stored by reseller and repeats for each reseller / consumer specialty combination. For queries that summarize sales by consumer specialty, the data totals are still inflated unless you can apply a distinct sum.
To help you accomplish this distinct sum, Analysis Services provides built-in support for many-to-many relationships. Once a many-to-many relationship is defined, Analysis Services can apply distinct sums to correctly aggregate data where necessary. To obtain this benefit in the reseller example, first create the Sales measure group that includes the Dim Reseller dimension and sales measures from the Fact Sales table. Next, you can use a many-to-many relationship to relate the Sales measure group to the Consumer Specialty many-to-many dimension. To set up this relationship, you must identify an intermediate measure group that can be used to map Sales data to Consumer Specialty. In this scenario, the intermediate measure group is Reseller Specialty, sourced from the Fact Reseller Specialty fact table.
Performance considerations
During processing, the data and intermediate measure groups are processed independently. Fact data and aggregations for the data measure group do not include any attributes from the many-to-many dimension. When you query the data measure group by the many-to-many dimension, a run-time “join” is performed between the two measure groups using the granularity attributes of each dimension that the measure groups have in common. In the example in Figure 26, when users want to query Sales data by Consumer Specialty, a run-time join is performed between the Sales measure group and Reseller Specialty measure group using the Reseller Key of the Reseller dimension. From a performance perspective, the run-time join has the greatest impact on query performance. More specifically, if the intermediate measure group is larger than the data measure group or if the many-to-many dimension is generally large (at least one million members), you can experience query performance issues due to the amount of data that needs to be joined at run time. To optimize the run-time join, review the aggregation design for the intermediate measure group to verify that aggregations include attributes from the many-to-many dimension. In the example in Figure 26, aggregations for the intermediate measure group should include attributes from the Consumer Specialty dimension such as the description attribute. While many-to-many relationships are very powerful, to avoid query performance issues, in general you should limit your use of many-to-many relationships to smaller intermediate measure groups and dimensions.
Reference relationships
In traditional dimension design scenarios, all dimension tables join directly to the fact table by means of their primary keys. In snowflake dimension designs, multiple dimension tables are chained together, with the chained dimensions joining indirectly to the fact table by means of a key in another dimension table. These chained dimensions are often called snowflake dimension tables. Figure 27 displays an example of snowflake dimension tables. Each table in the snowflake is linked to a subsequent table via a foreign key reference.
Figure 27 Snowflake dimension tables
In Figure 27, the Dim Reseller table has a snowflake relationship to the Dim Geography table. In addition, the Dim Customer table has a snowflake relationship to the Dim Geography table. From a relational point of view, if you want to analyze customer sales by geography, you must join Dim Geography to Fact Customer Sales via Dim Customer. If you want to analyze reseller sales by geography, you must join Dim Geography to Fact Reseller Sales via Dim Reseller.
Within Analysis Services, dimensions are “joined” to measure groups by specifying the dimension’s relationship type. Most dimensions have regular relationships where a dimension table is joined directly to the fact table. However, for snowflake dimension table scenarios, such as the one depicted in Figure 27, there are two general design techniques that you can adopt as described in this section.
For each dimension entity that joins to the fact table, create a single OLAP dimension that combines attributes from all of the snowflake dimension tables, and then join each dimension to measure group using a regular relationship type. If you have multiple dimension entities that reference the same snowflake tables, attributes from the shared snowflake tables are repeated across the OLAP dimensions.
To apply this technique to the Figure 27 example, create two OLAP dimensions: 1) a Reseller dimension that contains attributes from both the Dim Reseller and Dim Geography tables, and 2) a Customer dimension that contains attributes from Dim Customer and Dim Geography. Note that attributes from Dim Geography are repeated across both the Reseller and Customer dimensions.
For the Reseller Sales measure group, use a Regular relationship for the Reseller dimension. For the Customer Sales measure group, use a regular relationship Customer dimension. Remember that the relationship between a dimension and a measure group defines how the dimension data is to be “joined” to the fact data. A regular relationship means that you have defined a direct relationship between one or more dimension columns and one or more measure group columns.
With this design, for each OLAP dimension, all of the snowflake dimension tables are joined together at processing time and the OLAP dimension is materialized on disk. As with any other processing operation, you can control whether the processing should remove missing keys or use the unknown member for any records that do join across all tables.
From a performance perspective, the benefit of this approach is the ability to create natural hierarchies and use aggregations. Since each dimension has a regular relationship to the measure group, to enhance query performance, aggregations can be designed for attributes in each dimension, given the proper configuration of attribute relationships. In addition, during querying, you can take advantage of Autoexists optimizations that naturally occur between attributes within a dimension. For more information on Autoexists, see Removing empty tuples.
If you use this approach, you must also consider the impact of increasing the number of attributes that the aggregation design algorithm must consider. By repeating attributes across multiple dimensions, you are creating more work for the aggregation design algorithm which could negatively impact processing times.
Option 2 – Use a reference relationship
An alternative design approach to combining attributes involves reference relationships. Reference relationships allow you to indirectly relate OLAP dimensions to a measure group using an intermediate dimension. The intermediate dimension creates a “join” path that the measure group can use to relate its data to each reference dimension.
To apply this technique to the example in Figure 27, create three separate OLAP dimensions for customer, reseller, and geography. The following describes how these dimensions can be related to each measure group (Reseller Sales and Customer Sales):
- The Customer Sales measure group contains a regular relationship to the Customer dimension and contains a reference relationship to the Geography dimension. The reference relationship uses Customer as an intermediate dimension to assign sales to specific geographies.
- The Reseller Sales measure group contains a regular relationship to the Reseller dimension and contains a reference relationship to the Geography dimension. The reference relationship uses Reseller as an intermediate dimension to assign sales to specific geographies.
To use this technique effectively, you must consider the impacts of reference relationships on processing and query performance. During processing, each dimension is processed independently. No attributes from the reference dimension are automatically considered for aggregation. During querying, measure group data is joined to the reference dimension as necessary by means of the intermediate dimension. For example, if you query customer sales data by geography, a run-time join must occur from the Customer Sales measure group to Geography via the Customer dimension. This process can be somewhat slow for large dimensions. In addition, any missing attribute members that are encountered during querying are automatically assigned to the unknown member in order to preserve data totals.
To improve the query performance of reference relationships, you can choose to materialize them. Note that by default, reference relationships are not materialized. When a reference relationship is materialized, the joining across dimensions is performed during processing as opposed to querying. In addition, the attributes in the materialized reference dimensions follow the aggregation rules of standard dimensions. For more information on these rules, see The aggregation usage rules. Since the join is performed during processing and aggregations are possible, materialized reference relationships can significantly improve query performance when compared to unmaterialized relationships.
Some additional considerations apply to materialized reference relationships. During processing, the reference dimension is processed independently. At this time, if any row in the measure group does not join to the reference dimension, the record is removed from the partition. Note that this is different behavior than the unmaterialized reference relationship where missing members are assigned to the unknown member.
To better understand how missing members are handled for materialized relationships, consider the following example. If you have a sales order in the Customer Sales fact table that maps to a specific customer but that customer has a missing geography, the record cannot join to the Geography table and is rejected from the partition. Therefore, if you have referential integrity issues in your source data, materializing the reference relationship can result in missing data from the partition for those fact records that do not join to the reference dimension. To counteract this behavior and handle missing values, you can create your own unknown dimension record in the reference dimension table and then assign that value to all records missing reference values during your extraction, transformation, and loading (ETL) processes. With the unknown record in place for missing values, at processing time, all customer records can successfully join to the reference dimension.
Option Comparison – Combining attributes vs. using reference relationships
When you compare these two design alternatives, it is important to assess the overall impacts on processing and query performance. When you combine attributes, you can benefit from creating natural hierarchies and using aggregations to improve query performance. When you use reference relationships, a reference dimension can only participate in aggregations if it is materialized. These aggregations will not take into account any hierarchies across dimensions, since the reference dimension is analyzed separately from the other dimensions. In light of this information, the following guidelines can help you decide which approach to adopt:
- If your dimension is frequently queried and can benefit from natural hierarchies and aggregations, you should combine attributes from snowflake dimension tables into your normal dimension design.
- If the dimension is not frequently queried and is only used for one-off analysis, you can use unmaterialized reference relationships to expose the dimension for browsing without the overhead of creating aggregations for dimension attributes that are not commonly queried. If the intermediate dimensions are large, to optimize the query performance you can materialize the reference relationship.
As an additional design consideration, note that the example in Figure 27 includes snowflake tables with two fact tables / measure groups. When you have snowflake tables that join to a single fact table/ measure group, as depicted in Figure 28, the only available design option is to combine attributes.
Figure 28 One measure group
Reference relationships are not applicable to this design scenario because Analysis Services only allows one reference relationship per dimension per measure group. In the example in Figure 28, this means that when you define Dim Geography as a reference dimension to Reseller Sales, you must either select Dim Reseller or Dim Employee as the intermediate dimension. This either/or selection is not likely going to satisfy your business requirements. As a result, in this scenario, you can use design option 1, combining attributes, to model the snowflake dimension tables.
Near real-time data refreshes
Whenever you have an application that requires a low level of latency, such as in near real-time data refreshes, you must consider a special set of performance tuning techniques that can help you balance low levels of data latency with optimal processing and query performance.
Generally speaking, low levels of data latency include hourly, minute, and second refresh intervals. When you need to access refreshed data on an hourly basis, for example, your first instinct may be to process all of your dimensions and measure groups every hour. However, if it takes 30 minutes to process all database objects, to meet the hourly requirement, you would need to reprocess every 30 minutes. To further complicate this, you would also need to assess the performance impact of any concurrent querying operations that are competing for server resources.
Instead of processing all measure groups and dimensions to meet a low latency requirement, to improve performance and enhance manageability, you can use partitions to isolate the low latency data from the rest of the data. Typically this means that you create one or more near real-time partitions that are constantly being updated with refreshed data. Isolating the low latency data follows normal best practices for partitioning so that you can process and update the near real-time partitions without impacting the other partitions. Using this solution, you can also reduce the amount of data that needs to be processed in near real-time.
With partitions in place, the next step is to decide on how you are going to refresh the data in your near real-time partition(s). Remember that partitions require you to process them in order to refresh the data. Depending on the size of your near real-time partition(s) and the required frequency of the data updates, you can either schedule the refresh of a partition on a periodic basis or you can enable Analysis Services to manage the refresh of a partition using proactive caching.
Proactive caching is a feature in Analysis Services that transparently synchronizes and maintains a partition or dimension much like a cache. A proactive caching partition or dimension is commonly referred to as a cache, even though it is still considered to be a partition or dimension. Using proactive caching, you can enable Analysis Services to automatically detect a data change in the source data, to incrementally update or rebuild the cache with the refreshed data, and to expose the refreshed data to end users.
Even though proactive caching automates much of the refresh work for you, from a performance perspective, you must still consider the typical parameters that impact processing performance such as the frequency of source system updates, the amount of time it takes to update or rebuild the cache, and the level of data latency that end users are willing to accept.
From a performance perspective, there are three groups of settings that impact query responsiveness and processing performance for proactive caching: notification settings, refresh settings, and availability settings.
Notification settings
Notification settings impact how Analysis Services detects data changes in the source system. To satisfy the needs of different data architectures, Analysis Services provides a few mechanisms that you can use to notify Analysis Services of data changes. From a performance perspective, the Scheduled polling mechanism provides the most flexibility by allowing you to either rebuild or incrementally update the cache. Incremental updates improve proactive caching performance by reducing the amount of data that needs to be processed. For proactive caching partitions, incremental updates use a ProcessAdd to append new data to the cache. For proactive caching dimensions, a ProcessUpdate is performed. If you use Scheduled polling without incremental updates, the cache is always completely rebuilt.
Refresh settings
Refresh settings impact when Analysis Services rebuilds or updates the cache. The two most important refresh settings are Silence Interval and Silence Override Interval.
- Silence Interval defines how long Analysis Services waits from the point at which it detects a change to when it begins refreshing the cache. Since Analysis Services does not officially know when source data changes are finished, the goal is for Analysis Services to refresh the cache when there is a lull in the propagation of source data changes. The lull is defined by a period of silence or no activity that must expire before Analysis Services refreshes the cache. Consider the example where the Silence Interval is set to ten seconds and your cache is configured to be fully rebuilt during refresh. In this scenario, once Analysis Services detects a data change, a time counter starts. If ten seconds pass and no additional changes are detected, Analysis Services rebuilds the cache. If Analysis Services detects additional changes during that time period, the time counter resets each time a change is detected.
- Silence Override Interval determines how long Analysis Services waits before performing a forced refresh of the cache. Silence Override Interval is useful when your source database is updated frequently and the Silence Interval threshold cannot be reached, since the time counter is continuously reset. In this scenario, you can use the Silence Override Interval to force a refresh of the cache after a certain period of time, such as five minutes or ten minutes. For example, if you set the Silence Override Interval to ten minutes, every ten minutes Analysis Services refreshes the cache only if a change was detected.
Availability settings
Availability settings allow you to control how data is exposed to end users during cache refresh. If
your cache takes several minutes to update or rebuild, you may want to consider configuring the proactive caching settings to allow users to see an older cache until the refreshed cache is ready. For example, if you configure a partition to use Automatic MOLAP settings (Silence Interval = 10 seconds and Silence Override Interval = 10 minutes), during cache refresh, users query the old cache until the new cache is ready. If the cache takes five hours to rebuild, users must wait five hours for cache rebuild to complete. If you always want users to view refreshed data, enable the Bring Online Immediately setting. With Bring Online Immediately enabled, during cache refresh all queries are directed to the relational source database to retrieve the latest data for end users. While this provides users with refreshed data, it can also result in reduced query performance given that Analysis Services needs to redirect queries to the relational source database. If you want finer grained control over users viewing refreshed data, you can use the Latency setting to define a threshold that controls when queries are redirected to the source database during a cache refresh. For example, if you set Latency to four hours and the cache requires five hours to rebuild, for four hours, the queries will be satisfied by the older cache. After four hours, queries are redirected to the source database until the cache has completed its refresh.
Figure 29 illustrates how the proactive caching settings impact queries during cache rebuilding.
Figure 29 Proactive caching example
Generally speaking, to maximize query performance, it is a good practice to increase Latency where possible so that queries can continue to execute against the existing cache while data is read and processed into a new cache whenever the silence interval or silence override interval is reached. If you set the Latency too low, query performance may suffer as queries are continuously redirected to the relational source. Switching back and forth between the partition and the relational source can provide very unpredictable query response times for users. If you expect queries to constantly be redirected to the source database, to optimize query performance, you must ensure that Analysis Services understands the partition’s data slice. Setting a partition’s data slice is not necessary for traditional partitions. However, the data slice must be manually set for proactive caching partitions, as well as any ROLAP partition. In light of the potential redirection of queries to the relational source, proactive caching is generally not recommended on cubes that are based on multiple data sources.
If you enable proactive caching for your dimensions, to optimize processing, pay special attention to the Latency setting for each dimension and the overall impact of redirecting queries to the source database. When a dimension switches from the dimension cache to the relational database, all partitions that use the dimension need to be fully reprocessed. Therefore, where possible, it is a good idea to ensure that the Latency setting allows you to query the old cache until the new cache is rebuilt.
Tuning Server Resources
Query responsiveness and efficient processing require effective usage of memory, CPU, and disk resources. To control the usage of these resources, Analysis Services 2005 introduces a new memory architecture and threading model that use innovative techniques to manage resource requests during querying and processing operations.
To optimize resource usage across various server environments and workloads, for every Analysis Services instance, Analysis Services exposes a collection of server configuration properties. To provide ease-of-configuration, during installation of Analysis Services 2005, many of these server properties are dynamically assigned based on the server’s physical memory and number of logical processors. Given their dynamic nature, the default values for many of the server properties are sufficient for most Analysis Services deployments. This is different behavior than previous versions of Analysis Services where server properties were typically assigned static values that required direct modification. While the Analysis Services 2005 default values apply to most deployments, there are some implementation scenarios where you may be required to fine tune server properties in order to optimize resource utilization.
Regardless of whether you need to alter the server configuration properties, it is always a best practice to acquaint yourself with how Analysis Services uses memory, CPU, and disk resources so you can evaluate how resources are being utilized in your server environment.
For each resource, this section presents two topics: 1) a topic that describes how Analysis Services uses system resources during querying and processing, and 2) a topic that organizes practical guidance on the design scenarios and data architectures that may require the tuning of additional server properties.
Understanding how Analysis Services uses memory – Making the best performance decisions about memory utilization requires understanding how the Analysis Services server manages memory overall as well as how it handles the memory demands of processing and querying operations.
Optimizing memory usage – Optimizing memory usage requires applying a series of techniques to detect whether you have sufficient memory resources and to identify those configuration properties that impact memory resource utilization and overall performance.
Understanding how Analysis Services uses CPU resources – Making the best performance decisions about CPU utilization requires understanding how the Analysis Services server uses CPU resources overall as well as how it handles the CPU demands of processing and querying operations.
Optimizing CPU usage – Optimizing CPU usage requires applying a series of techniques to detect whether you have sufficient processor resources and to identify those configuration properties that impact CPU resource utilization and overall performance.
Understanding how Analysis Services uses disk resources – Making the best performance decisions about disk resource utilization requires understanding how the Analysis Services server uses disk resources overall as well as how it handles the disk resource demands of processing and querying operations.
Optimizing disk usage – Optimizing disk usage requires applying a series of techniques to detect whether you have sufficient disk resources and to identify those configuration properties that impact disk resource utilization and overall performance.
Understanding how Analysis Services uses memory
Analysis Services 2005 introduces a new memory architecture that allocates and manages memory in a more efficient manner than previous versions of Analysis Services where the memory management of dimensions and other objects placed limits on querying and processing performance. The primary goal of memory management in Analysis Service 2005 is to effectively utilize available memory while balancing the competing demands of processing and querying operations.
To help you gain some familiarity with the distinct memory demands of these two operations, following is a high-level overview of how Analysis Services 2005 uses memory during processing and querying:
- Querying—During querying, memory is used at various stages of query execution to satisfy query requests. To promote fast data retrieval, the Storage Engine cache is used to store measure group data. To promote fast calculation evaluation, the Query Execution Engine cache is used to store calculation results. To efficiently retrieve dimensions, dimension data is paged into memory as needed by queries, rather than being loaded into memory at server startup, as in prior versions of Analysis Services. To improve query performance, it is essential to have sufficient memory available to cache data results, calculation results, and as-needed dimension data.
- Processing—During processing, memory is used to temporarily store, index, and aggregate data before writing to disk. Each processing job requests a specific amount of memory from the Analysis Services memory governor. If sufficient memory is not available to perform the job, the job is blocked. To ensure that processing proceeds in an efficient manner, it is important to verify that there is enough memory available to successfully complete all processing jobs and to optimize the calculation of aggregations in memory.
Given the distinct memory demands of querying and processing, it is important to not only understand how each operation impacts memory usage, but it is also important to understand how the Analysis Services server manages memory across all server operations. The sections that follow describe the memory management techniques of the Analysis Services server as well as how it handles the specific demands of querying and processing
Memory management
To effectively understand Analysis Services memory management techniques, you must first consider the maximum amount of memory that Analysis Services can address. Analysis Services relies on Microsoft Windows virtual memory for its memory page pool. The amount of memory it can address depends on the version of SQL Server that you are using:
- For SQL Server 2005 (32-bit), the maximum amount of virtual memory that an Analysis Services process can address is 3 gigabytes (GB). By default, an Analysis Services process can only address 2 GB; however it is possible to enable Analysis Services to address 3 GB. For guidelines on how to optimize the addressable memory of Analysis Services, see Increasing available memory.
- SQL Server 2005 (64-bit) is not limited by a 3-GB virtual address space limit, enabling the 64-bit version of Analysis Services to use as much address space as it needs. This applies to both IA64 and X64 architectures.
To perform server operations, Analysis Services requests allocations of memory from the Windows operating system, and then returns that memory to the Windows operating system when the allocated memory is no longer needed. Analysis Services manages the amount of memory allocated to the server by using a memory range that is defined by two server properties: Memory\TotalMemoryLimit and Memory\LowMemoryLimit.
- Memory\TotalMemoryLimit represents the upper limit of memory that the server uses to manage all Analysis Services operations. If Memory\TotalMemoryLimit is set to a value between 0 and 100, it is interpreted as a percentage of total physical memory. If the property is set to a value above 100, Analysis Services interprets it as an absolute memory value in bytes. The default value for Memory\TotalMemoryLimit is 80, which translates to 80% of the amount of physical memory of the server.
Note that this property does not define a hard limit on the amount of memory that Analysis Services uses. Rather, it is a soft limit that is used to identify situations where the server is experiencing memory pressure. For some operations, such as processing, if Analysis Services requires additional memory beyond the value of Memory\TotalMemoryLimit, the Analysis Services server attempts to reserve that memory regardless of the value of the property.
- Memory\LowMemoryLimit represents the lower limit of memory that the server uses to manage all Analysis Services operations. Like the Memory\TotalMemoryLimit property, a value between 0 and 100 is interpreted as a percentage of total physical memory. If the property is set to a value above 100, Analysis Services interprets it as an absolute memory value in bytes.
The default value for the Memory\LowMemoryLimit property is 75, which translates to 75% of the amount of physical memory on the Analysis Services server.
Analysis Services uses these memory range settings to manage how memory is allocated and used at various levels of memory pressure. When the server experiences elevated levels of memory pressure, Analysis Services uses a set of cleaner threads, one cleaner thread per logical processor, to control the amount of memory allocated to various operations. Depending on the amount of memory pressure, the cleaner threads are activated in parallel to shrink memory as needed. The cleaner threads clean memory according to three general levels of memory pressure:
- No Memory Pressure—If the memory used by Analysis Services is below the value set in the Memory\LowMemoryLimit property, the cleaner does nothing.
- Some Memory Pressure—If the memory used by Analysis Services is between the values set in the Memory\LowMemoryLimit and the Memory\TotalMemoryLimit properties, the cleaner begins to clean memory using a cost/benefit algorithm. For more information on how the cleaner shrinks memory, see Shrinkable vs. non-shrinkable memory.
- High Memory Pressure—If the memory used by Analysis Services is above the value set in the Memory\TotalMemoryLimit property, the cleaner cleans until the memory used by Analysis Services reaches the Memory\TotalMemoryLimit. When the memory used by Analysis Services exceeds the Memory\TotalMemoryLimit, the server goes into an aggressive mode where it cleans everything that it can. If the memory used is mostly non-shrinkable (more information on non-shrinkable memory is included in the next section), and cannot be purged, Analysis Services detects that the cleaner was unable to clean much. If it is in this aggressive mode of cleaning, it tries to cancel active requests. When this point is reached, you may see poor query performance, out of memory errors in the event log, and slow connection times.
Shrinkable vs. non-shrinkable memory
Analysis Services divides memory into two primary categories: shrinkable memory and non-shrinkable memory as displayed in Figure 30.
Figure 30 Shrinkable vs. non-shrinkable memory
When the cleaner is activated, it begins evicting elements of shrinkable memory, based on a cost/benefit algorithm that takes into account a variety of factors, including how frequently the entry is used, the amount of resources required to resolve the entries, and how much space is consumed by related entries. Shrinkable memory elements include the following:
- Cached Results—Cached results include the Storage Engine data cache and Query Execution Engine calculation cache. As stated earlier in this document, the Storage Engine data cache contains measure group data and the Query Execution Calculation Engine cache contains calculation results. While both caches can help improve query response times, the data cache provides the most benefit to query performance by storing data that has been cached from disk. In situations of memory pressure, the cleaner shrinks the memory used for cached results. With this in mind, it is a good practice to monitor the usage of memory so that you can minimize the scenarios where elevated levels of memory pressure force the removal of cached results. For more information on how to monitor memory pressure, see Monitoring memory management.
- Paged in dimension data—Dimension data is paged in from the dimension stores as needed. The paged-in data is kept in memory until the cleaner is under memory pressure to remove it. Note that this is different behavior than previous versions of Analysis Services where all dimension data was resident in memory.
- Expired Sessions—Idle client sessions that have exceeded a longevity threshold are removed by the cleaner based on the level of memory pressure. Several server properties work together to manage the longevity of idle sessions. For more information on how to evaluate these properties, see Monitoring the timeout of idle sessions.
Non-shrinkable memory elements are not impacted by the Analysis Services cleaner. Non-shrinkable memory includes the following components:
- Metadata For each Analysis Services database, metadata is initialized and loaded into memory on demand. Metadata includes the definition of all objects in the database (not the data elements). The more objects in your database (including cubes, measure groups, partitions, and dimensions) and the more databases that you have on a given server, the larger the metadata overhead in memory. Note that this overhead is generally not large for most implementations. However, you can experience significant overhead if your Analysis Services server contains hundreds of databases with tens or hundreds of objects per database, such as in hosted solutions. For more information on how to monitor metadata overheard, see Minimizing metadata overhead.
- Active Sessions—For each active session, calculated members, named sets, connections, and other associated session information is retained as non-shrinkable memory.
- Query Memory and Process Memory—Analysis Services reserves specific areas of memory for temporary use during querying and processing. During the execution of a query, for example, memory may be used to materialize data sets such as during the cross joining of data. During processing, memory is used to temporarily store, index, and aggregate data before it are written to disk. These memory elements are non-shrinkable because they are only needed to complete a specific server operation. As soon as the operation is over, these elements are removed from memory.
Memory demands during querying
During querying, memory is primarily used to store cached results in the data and calculation caches. As stated previously, of the two caches, the one that provides the most significant performance benefit is the data cache. When Analysis Services first starts, the data cache is empty. Until the data cache is loaded with data from queries, Analysis Services must resolve user queries by using data stored on disk, either by scanning the fact data or by using aggregations. Once these queries are loaded into the data cache, they remain there until the cleaner thread removes them or the cache is flushed during measure group or partition processing.
You can often increase query responsiveness by preloading data into the data cache by executing a generalized set of representative user queries. This process is called cache warming. While cache warming can be a useful technique, cache warming should not be used as a substitute for designing and calculating an appropriate set of aggregations. For more information on cache warming, see Warming the data cache.
Memory demands during processing
During processing, memory is required to temporarily store fact data and aggregations prior to writing them to disk.
Processing Fact data
Processing uses a double-buffered scheme to read and process fact records from the source database. Analysis Services populates an initial buffer from the relational database, and then populates a second buffer from the initial buffer where the data is sorted, indexed, and written to the partition file in segments. Each segment consists of 65,536 rows; the number of bytes in each segment varies based on the size of each row.
The OLAP\Process\BufferMemoryLimit property controls the amount of memory that is used per processing job to store and cache data coming from a relational data source. This setting along with OLAP\Process\BufferRecordLimit determines the number of rows that can be processed in the buffers. The OLAP\Process\BufferMemoryLimit setting is interpreted as a percentage of total physical memory if the value is less than 100, or an absolute value of bytes if the value is greater than 100. The default value is 60, which indicates that a maximum of 60% of the total physical memory can be used. For most deployments, the default value of OLAP\Process\BufferMemoryLimit provides sufficient processing performance. For more information on scenarios where it may be appropriate to change the value of this property, see Tuning memory for partition processing.
Building Aggregations
Analysis Services uses memory during the building of aggregations. Each partition has its own aggregation buffer space limit. Two properties control the size of the aggregation buffer:
- OLAP\Process\AggregationMemoryLimitMax is a server property that controls the maximum amount of memory that can be used for aggregation processing per partition processing job. This value is interpreted as a percentage of total memory if the value is less than 100, or an absolute value of bytes if the value is greater than 100. The default value is 80, which indicates that a maximum of 80% of the total physical memory can be used for aggregation processing.
- OLAP\Process\AggregationMemoryLimitMin is a server property that controls the minimum amount of memory that can be used for aggregation processing per partition processing job. This value is interpreted as a percentage of total memory if the value is less than 100, or an absolute value of bytes if the value is greater than 100. The default value is 10, which indicates that a minimum of 10% of the total physical memory will be used for aggregation processing
For a given partition, all aggregations are calculated at once. As a general best practice, it is a good idea to verify that all aggregations can fit into memory during creation; otherwise temporary files are used, which can slow down processing although the impact on performance is not as significant as in prior versions of Analysis Services. For more information on how to monitor the usage of temporary files, see Tuning memory for partition processing.
Optimizing memory usage
Optimizing memory usage for querying and processing operations requires supplying the server with adequate memory and verifying that the memory management properties are configured properly for your server environment. This section contains a summary of the guidelines that can help you optimize the memory usage of Analysis Services.
Increasing available memory
If you have one or more large or complex cubes and are using SQL Server 2005 (32-bit), use Windows Advanced Server® or Datacenter Server with SQL Server 2005 Enterprise Edition (or SQL Server 2005 Developer Edition) to enable Analysis Services to address up to 3 GB of memory. Otherwise, the maximum amount of memory that Analysis Services can address is 2 GB.
To enable Analysis Services to address more than 2 GB of physical memory with either of these editions, enable the Application Memory Tuning feature of Windows. To accomplish this, use the /3GB switch in the boot.ini file If you set the /3GB switch in the boot.ini file, the server should have at least 4 GB of memory to ensure that the Windows operating system also has sufficient memory for system services. If you run other applications on the server, you must factor in their memory requirements as well.
If you have one or more very large and complex cubes and your Analysis Services memory needs cannot be met within the 3-GB address space, SQL Server 2005 (64-bit) allows the Analysis Services process to access more than 3 GB of memory. You may also want to consider SQL Server 2005 (64-bit) in design scenarios where you have many partitions that you need to process in parallel or large dimensions that require a large amount of memory to process.
If you cannot add additional physical memory to increase performance, increasing the size of the paging files on the Analysis Services server can prevent out–of-memory errors when the amount of virtual memory allocated exceeds the amount of physical memory on the Analysis Services server.
Monitoring memory management
Given that the Memory\TotalMemoryLimit and Memory\LowMemoryLimit
properties are percentages by default, they dynamically reflect the amount of physical memory on the server, even if you add new memory to the server. Using these default percentages is beneficial in deployments where Analysis Services is the only application running on your server.
If Analysis Services is installed in a shared application environment, such as if you have Analysis Services installed on the same machine as SQL Server, consider assigning static values to these properties as opposed to percentages in order to constrain Analysis Services memory usage. In shared application environments, it is also a good idea to constrain how other applications on the server use memory.
When modifying these properties, it is a good practice to keep the difference between the Memory\LowMemoryLimit and Memory\TotalMemoryLimit is at least five percent, so that the cleaner can smoothly transition across different levels of memory pressure.
You can monitor the memory management of the Analysis Services server by using the following performance counters displayed in Table 3.
Table 3 Memory management performance counters
|
Performance counter name |
Definition |
|
MSAS 2005:Memory\Memory Limit Low KB |
Displays the Memory\LowMemoryLimit from the configuration file |
|
MSAS 2005:Memory\Memory Limit High KB |
Displays the Memory\TotalMemoryLimit from the configuration file. |
|
MSAS 2005:Memory\Memory Usage KB |
Displays the memory usage of the server process. This is the value that is compared to Memory\LowMemoryLimit and Memory\TotalMemoryLimit. Note that the value of this performance counter is the same value displayed by the Process\Private Bytes performance counter. |
|
MSAS 2005:Memory\Cleaner Balance/sec |
Shows how many times the current memory usage is compared against the settings. Memory usage is checked every 500ms, so the counter will trend towards 2 with slight deviations when the system is under high stress. |
|
MSAS 2005:Memory\Cleaner Memory nonshrinkable KB |
Displays the amount of memory, in KB, non subject to purging by the background cleaner. |
|
MSAS 2005:Memory\Cleaner Memory shrinkable KB |
Displays the amount of memory, in KB, subject to purging by the background cleaner. |
|
MSAS 2005:Memory\Cleaner Memory KB |
Displays the amount of memory, in KB, known to the background cleaner. (Cleaner memory shrinkable + Cleaner memory non-shrinkable.) Note that this counter is calculated from internal accounting information so there may be some small deviation from the memory reported by the operating system. |
Minimizing metadata overhead
For each database, metadata is initialized and loaded into non-shrinkable memory. Once loaded into memory, it is not subject to purging by the cleaner thread.
To monitor the metadata overhead for each database
- Restart the Analysis Services server.
- Note the starting value of the MSAS 2005:Memory\Memory Usage KB performance counter.
- Perform an operation that forces metadata initialization of a database, such as iterating over the list of objects in AMO browser sample application, or issuing a backup command on the database.
- After the operation has completed, note the ending value of MSAS 2005:Memory\Memory Usage KB. The difference between the starting value and the ending value represents the memory overhead of the database.
For each database, you should see memory growth proportional to the number of databases you initialize. If you notice that a large amount of your server memory is associated with metadata overhead, you may want consider whether you can take steps to reduce the memory overhead of a given database. The best way to do this is to re-examine the design of the cube. An excessive number of dimensions attributes or partitions can increase the metadata overhead. Where possible you should follow the design best practices outlined in Optimizing the dimension design and Reducing attribute overhead.
Monitoring the timeout of idle sessions
Client sessions are managed in memory. In general, there is a one-to-one relationship between connections and sessions. While each connection consumes approximately 32 KB of memory, the amount of memory a given session consumes depends on the queries and calculations performed in that session. You can monitor the current number of user sessions and connections by using the MSAS 2005:Connection\Current
connections and MSAS 2005:Connection\Current
user sessions performance counters to evaluate the connection and session demands on your system.
As stated earlier, active sessions consume non-shrinkable memory, whereas expired sessions consume shrinkable memory. Two main properties determine when a session expires:
- The MinIdleSessionTimeout is the threshold of idle time in seconds after which the server can destroy a session based on the level of memory pressure. The MinIdleSessionTimeout is set to 2,700 seconds (45 minutes). This means that a session must be idle for 45 minutes before it is considered an expired session that the cleaner can remove when memory pressure thresholds are exceeded.
- The MaxIdleSessionTimeout is the time in seconds after which the server forcibly destroys an idle session regardless of memory pressure. By default, the MaxIdleSessionTimeout is set to zero seconds, meaning that the idle session is never forcibly removed by this setting.
In addition to the properties that manage expired sessions, there are properties that manage the longevity of sessions that lose their connection. A connectionless session is called an orphaned session.
- The IdleOrphanSessionTimeout is a server property of that controls the timeout of connection-less sessions. By default this property is set to 120 seconds (three minutes), meaning that if a session loses its connection and a reconnection is not made within 120 seconds, the Analysis Services server forcibly destroys this session.
- IdleConnectionTimeout controls the timeout of connections that have not been used for a specified amount of time. By default, this property is set to zero seconds. This means that the connection never times out. However, given that the connection cannot exist outside of a session, any idle connection will be cleaned up whenever its session is destroyed.
For most scenarios, these default settings provide adequate server management of sessions. However, there may be scenarios where you want finer-grained session management. For example, you may want to alter these settings according to the level amount of memory pressure that the Analysis Services server is experiencing. During busy periods of elevated memory pressure, you may want to destroy idle sessions after 15 minutes. At times when the server is not busy, you want the idle sessions to be destroyed after 45 minutes. To accomplish this, set the MinIdleSessionTimeout property to 900 seconds (15 minutes) and the MaxIdleSessionTimeout to 2,700 seconds.
Note that before changing these properties, it is important to understand how your client application manages sessions and connections. Some client applications, for example, have their own timeout mechanisms for connections and sessions that are managed independently of Analysis Services.
Tuning memory for partition processing
Tuning memory for partition processing involves three general techniques:
- Modifying the OLAP\Process\BufferMemoryLimit property as appropriate.
- Verifying that sufficient memory is available for building aggregations.
- Splitting up processing jobs in memory-constrained environments.
Modifying the OLAP\Process\BufferMemoryLimit property as appropriate
OLAP\Process\BufferMemoryLimit determines the size of the fact data buffers using during partition processing. While the default value of the OLAP\Process\BufferMemoryLimit is sufficient for many deployments, you may find it useful to alter the property in the following scenarios:
- If the granularity of your measure group is more summarized than the relational source fact table, generally speaking you may want to consider increasing the size of the buffers to facilitate data grouping. For example, if the source data has a granularity of day and the measure group has a granularity of month, Analysis Services must group the daily data by month before writing to disk. This grouping only occurs within a single buffer and it is flushed to disk once it is full. By increasing the size of the buffer, you decrease the number of times that the buffers are swapped to disk and also decrease the size of the fact data on disk, which can also improve query performance.
- If the OLAP measure group is of the same granularity as the source relational fact table, you may benefit from using smaller buffers. When the relational fact table and OLAP measure group are at roughly the same level of detail, there is no need to group the data, because all rows remain distinct and cannot be aggregated. In this scenario, assigning smaller buffers is helpful, allowing you to execute more processing jobs in parallel.
Verifying that sufficient memory is available for building aggregations
During processing, the aggregation buffer determines the amount of memory that is available to build aggregations for a given partition. If the aggregation buffer is too small, Analysis Services supplements the aggregation buffer with temporary files. Temporary files are created in the TempDir folder when memory is filled and data is sorted and written to disk. When all necessary files are created, they are merged together to the final destination. Using temporary files can potentially result in some performance degradation during processing; however, the impact is generally not significant given that the operation is simply an external disk sort. Note that this behavior is different than in previous versions of Analysis Services.
To monitor any temporary files used during processing, review the MSAS 2005:Proc Aggregations\Temp file bytes written/sec or the MSAS 2005:Proc Aggregations\Temp file rows written/sec performance counters.
In addition, when processing multiple partitions in parallel or processing an entire cube in a single transaction, you must ensure that the total memory required does not exceed the Memory\TotalMemoryLimit property. If Analysis Services reaches the Memory\TotalMemoryLimit
during processing, it does not allow the aggregation buffer to grow and may cause temporary files to be used during aggregation processing. Furthermore, if you have insufficient virtual address space for these simultaneous operations, you may receive out-of-memory errors. If you have insufficient physical memory, memory paging will occur. If processing in parallel and you have limited resources, consider doing less in parallel.
Splitting up processing jobs in memory-constrained environments
During partition processing in memory-constrained environments, you may encounter a scenario where a ProcessFull operation on a measure group or partition cannot proceed due to limited memory resources. What is happening in this scenario is that the Process job requests an estimated amount of memory to complete the total ProcessFull operation. If the Analysis Services memory governor cannot secure enough memory for the job, the job can either fail or block other jobs as it waits for more memory to become available. As an alternative to performing a ProcessFull, you can split the processing operation into two steps by performing two operations serially: ProcessData and ProcessIndexes. In this scenario, the memory request will be smaller for each sequential operation and is less likely to exceed the limits of the system resources.
Warming the data cache
During querying, memory is primarily used to store cached results in the data and calculation caches. To optimize the benefits of caching, you can often increase query responsiveness by preloading data into the data cache by executing a generalized set of representative user queries. This process is called cache warming. To do this, you can create an application that executes a set of generalized queries to simulate typical user activity in order to expedite the process of populating the query results cache. For example, if you determine that users are querying by month and by product, you can create a set of queries that request data by product and by month. If you run this query whenever you start Analysis Services, or process the measure group or one of its partitions, this will pre-load the query results cache with data used to resolve these queries before users submit these types of query. This technique substantially improves Analysis Services response times to user queries that were anticipated by this set of queries.
To determine a set of generalized queries, you can use the Analysis Services query log to determine the dimension attributes typically queried by user queries. You can use an application, such as a Microsoft Excel macro, or a script file to warm the cache whenever you have performed an operation that flushes the query results cache. For example, this application could be executed automatically at the end of the cube processing step.
Running this application under an identifiable user name enables you to exclude that user name from the Usage-Based Optimization Wizard’s processing and avoid designing aggregations for the queries submitted by the cache warming application.
When testing the effectiveness of different cache-warming queries, you should empty the query results cache between each test to ensure the validity of your testing. You can empty the results cache using a simple XMLA command such as the following:
<Batch
xmlns=“http://schemas.microsoft.com/analysisservices/2003/engine“>
<ClearCache>
<Object>
<DatabaseID>Adventure Works DW</DatabaseID>
</Object>
</ClearCache>
</Batch>
This example XMLA command clears the cache for the Adventure Works DW database. To execute the ClearCache statement, you can either manually run the XMLA statement in SQL Server Management Studio or use the ASCMD tool command-line utility to execute any XMLA script.
Understanding how Analysis Services uses CPU resources
Analysis Services uses processor resources for both querying and processing. Increasing the number and speed of processors can significantly improve processing performance and, for cubes with a large number of users, improve query responsiveness as well.
Job architecture
Analysis Services uses a centralized job architecture to implement querying and processing operations. A job itself is a generic unit of processing or querying work. A job can have multiple levels of nested child jobs depending on the complexity of the request.
During processing operations, for example, a job is created for the object that you are processing, such as a dimension. A dimension job can then spawn several child jobs that process the attributes in the dimension. During querying, jobs are used to retrieve fact data and aggregations from the partition to satisfy query requests. For example, if you have a query that accesses multiple partitions, a parent job is generated for the query itself along with one or more child jobs per partition.
Generally speaking, executing more jobs in parallel has a positive impact on performance as long as you have enough processor resources to effectively handle the concurrent operations as well as sufficient memory and disk resources. The maximum number of jobs that can execute in parallel across all server operations (including both processing and querying) is determined by the CoordinatorExecutionMode property.
- A negative value for CoordinatorExecutionMode specifies the maximum number of parallel jobs that can start per processor.
- A value of zero enables the server to automatically determine the maximum number of parallel operations, based on the workload and available system resources.
- A positive value specifies an absolute number of parallel jobs that can start per server.
The default value for the CoordinatorExecutionMode is -4, which indicates that four jobs will be started in parallel per processor. This value is sufficient for most server environments. If you want to increase the level of parallelism in your server, you can increase the value of this property either by increasing the number of jobs per processor or by setting the property to an absolute value. While this globally increases the number of jobs that can execute in parallel, CoordinatorExecutionMode is not the only property that influences parallel operations. You must also consider the impact of other global settings such as the MaxThreads server properties that determine the maximum number of querying or processing threads that can execute in parallel. In addition, at a more granular level, for a given processing operation, you can specify the maximum number of processing tasks that can execute in parallel using the MaxParallel command. These settings are discussed in more detail in the sections that follow.
Thread pools
To effectively manage processor resources for both querying and processing operations, Analysis Services 2005 uses two thread pools:
- Querying thread pool—The querying thread pool controls the worker threads used by the Query Execution Engine to satisfy query requests. One thread from the querying pool is used per concurrent query. The minimum number of threads from the querying pool is determined by the value of the ThreadPool\Query\MinThreads property; its default setting is 1. The maximum number of worker threads maintained in the querying thread pool is determined by the value of ThreadPool\Query\MaxThreads; its default setting is 10.
- Processing thread pool—The processing thread pool controls the worker threads used by the Storage Engine during processing operations. The processing thread pool is also used during querying to control the threads used by the Storage Engine to retrieve data from disk. The ThreadPool\Process\MinThreads property determines the minimum number of processing threads that can be maintained at a given time. The default value of this property is 1. The ThreadPool\Process\MaxThreads property determines the maximum number of processing threads that can be maintained at a given time. The default value of this property is 64.
For scenarios on when these values should be changed, see Optimizing CPU usage. Before you modify these properties, it is useful to examine how these threads are used during querying and processing.
Processor demands during querying
During querying, to manage client connections, Analysis Services uses a listener thread to broker requests and create new server connections as needed. To satisfy query requests, the listener thread manages worker threads in the querying thread pool and the processing thread pool, assigns worker threads to specific requests, initiates new worker threads if there are not enough active worker threads in a given pool, and terminates idle worker threads as needed.
To satisfy a query request, the thread pools are used as follows:
- Worker threads from the query pool check the data and calculation caches respectively for any data and/or calculations pertinent to a client request.
- If necessary, worker threads from the processing pool are allocated to retrieve data from disk.
- Once data is retrieved, worker threads from the querying pool store the results in the query cache to resolve future queries.
- Worker threads from the querying pool perform necessary calculations and use a calculation cache to store calculation results.
The more threads that are available to satisfy queries, the more queries that you can execute in parallel. This is especially important in scenarios where you have a large number of users issuing queries. For more information on how to optimize processor resources during querying, see Maximize parallelism during querying.
Processor demands during processing
Where possible, Analysis Services naturally performs all processing operations in parallel. For every processing operation, you can specify the parallelism of the Analysis Services object by using the MaxParallel processing command. By default, the MaxParallel command is configured to Let the server decide, which is interpreted as unlimited parallelism, constrained only by hardware and server workload. For more information on how you can change this setting, see Maximize parallelism during processing.
Of all of the processing operations, partitions place the largest demands on processor resources. Each partition is processed in two stages and each stage is a multithreaded activity.
- During the first stage of processing a partition, Analysis Services populates an initial buffer from the relational database, populates a second buffer from the initial buffer, and then writes segments to the partition file. Analysis Services utilizes multiple threads for this stage, which execute asynchronously. This means that while data is being added to the initial buffer, data is being moved from the initial buffer into the second buffer and sorted into segments. When a segment is complete, it is written to the partition file. Processor usage during this first phase depends on the speed of the data transfer from the relational tables. Generally this stage is not particularly processor-intensive, using less than one processor. Rather, this stage is generally limited by the speed of retrieving data from the relational database. The maximum size of the buffer used to store the source data is determined by the OLAP\Process\BufferMemoryLimit and OLAP\Process\BufferRecordLimit server
properties. In some scenarios, it can be beneficial to modify these settings to improve processing performance. For more information on these properties, see Memory demands during processing.
- During the second stage, Analysis Services creates and computes aggregations for the data. Analysis Services utilizes multiple threads for this stage, executing these tasks asynchronously. These threads read the fact data into an aggregation buffer. If sufficient memory is allocated to the aggregation buffer, these aggregations are calculated entirely in memory. As stated previously in the Memory demands during processing section, if Analysis Services does not have sufficient memory to calculate aggregations, Analysis Services uses temporary files to supplement the aggregation buffer. This stage can be processor-intensive; Analysis Services takes advantage of multiple processors if they are available.
Optimizing CPU usage
While adding additional processor resources can improve the overall performance of Analysis Services, use the following guidelines to optimize the usage of processor resources.
Maximize parallelism during querying
As stated in the Thread pools section, Threadpool\Query\MaxThreads determines the maximum number of worker threads maintained in the querying thread pool. The default value of this property is 10. For servers that have more than one processor, to increase parallelism during querying, consider modifying Threadpool\Query\MaxThreads to be a number dependent on the number of server processors. A general recommendation is to set the Threadpool\Query\MaxThreads to a value of less than or equal to 2 times the number of processors on the server. For example, if you have an eight-processor machine, the general guideline is to set this value to no more than 16. In practical terms, increasing Threadpool\Query\MaxThreads will not significantly increase the performance of a given query. Rather, the benefit of increasing this property is that you can increase the number of queries that can be serviced concurrently.
Since querying also involves retrieving data from partitions, to improve parallel query operations, you must also consider the maximum threads available in the processing pool as specified by the Threadpool\Process\MaxThreads property. By default, this property has a value of 64. While partitions are naturally queried in parallel, when you have many queries that require data from multiple partitions, you can enhance data retrieval by changing the Threadpool\Process\MaxThreads property. When modifying this property, a general recommendation is to set the Threadpool\Process\MaxThreads to a value of less than or equal to 10 times the number of processors on the machine. For example, if you have an eight-processor server, the general guideline is setting this value to no more than 80. Note even though the default value is 64, if you have fewer than eight processors on a given server, you do not need to reduce the default value to throttle parallel operations. As you consider the scenarios for changing the Threadpool\Process\MaxThreads property, remember that changing this setting impacts the processing thread pool for both querying and processing. For more information on how this property specifically impacts processing operations, see Maximizing parallelism during processing.
While modifying the Threadpool\Process\MaxThreads and Threadpool\Query\MaxThreads properties can increase parallelism during querying, you must also take into account the additional impact of the CoordinatorExecutionMode. Consider the following example. If you have a four-processor server and you accept the default CoordinatorExecutionMode setting of
-4, a total of 16 jobs can be executed at one time across all server operations. So if ten queries are executed in parallel and require a total of 20 jobs, only 16 jobs can launch at a given time (assuming that no processing operations are being performed at that time). When the job threshold has been reached, subsequent jobs wait in a queue until a new job can be created. Therefore, if the number of jobs is the bottleneck to the operation, increasing the thread counts may not necessarily improve overall performance.
In practical terms, the balancing of jobs and threads can be tricky. If you want to increase parallelism, it is important to assess your greatest bottleneck to parallelism, such as the number of concurrent jobs and/or the number of concurrent threads, or both. To help you determine this, it is helpful to monitor the following performance counters:
- MSAS 2005: Threads\Query pool job queue length—The number of jobs in the queue of the query thread pool. A non-zero value means that the number of query jobs has exceeded the number of available query threads. In this scenario, you may consider increasing the number of query threads. However, if CPU utilization is already very high, increasing the number of threads will only add to context switches and degrade performance.
-
MSAS 2005: Threads\Query pool busy threads—The number of busy threads in the query thread pool.
- MSAS 2005: Threads\Query pool idle threads—The number of idle threads in the query thread pool.
Maximize parallelism during processing
For processing operations, you can use the following mechanisms to maximize parallelism:
- CoordinatorExecutionMode—As stated earlier, this server-wide property controls the number of parallel operations across the server. If you are performing processing at the same time as querying, it is a good practice to increase this value.
- Threadpool\Process\MaxThreads—Also discussed earlier in this section, this server-wide property increases the number of threads that can be used to support parallel processing operations.
-
MaxParallel processing command—Rather than globally specifying the number of parallel operations for a given Analysis Services instance, for every processing operation, you can specify the maximum number of tasks that can operate in parallel. In many scenarios, this is the most common setting that is used to affect parallelism. You can specify the MaxParallel command in two ways: the Maximum parallel tasks option in the processing user interface or a custom XMLA script.
-
Maximum parallel tasks option—When you launch a processing operation from SQL Server Management Studio or Business Intelligence Development Studio, you can specify the Maximum parallel tasks option to change the level of parallelism for a given processing operation as displayed in Figure 31. The default value of this setting is Let the server decide,
which is interpreted as unlimited parallelism, constrained only by hardware and server workload. The drop-down list displays a list of suggested values but you can specify any value. If you increase this value to increase parallelism, be wary of setting the property too high. Performing too many parallel operations at once can be counterproductive if it causes context switching and degrades performance.
Figure 31 Maximum parallel tasks setting
- Custom XMLA script —As an alternative to specifying the MaxParallel command in the user interface, you can write a custom XMLA script to perform a processing operation and use the MaxParallel element to control the number of parallel operations within the XMLA script.
-
When processing multiple partitions in parallel, use the guidelines displayed in Table 4 for the number of partitions that can be processed in parallel according to the number of processors. These guidelines were taken from processing tests performed using Project REAL cubes.
Table 4 Partition processing guidelines
|
# of Processors |
# of Partitions to be processed in parallel |
|
4 |
2 – 4 |
|
8 |
4 – 8 |
|
16 |
6 – 16 |
Note that the actual number of partitions that can be processed in parallel depends on the querying workload and design scenario. For example, if you are performing querying and processing at the same time, you may want to decrease the number of partitions processed in parallel in order to keep some free resources for querying. Alternatively, if your design contains SQL queries with many complex joins, your parallel partition processing performance could be limited by the source database. If the source database is on the same machine as Analysis Services, you may see memory and CPU interactions that limit of the benefits of parallel operations. In fact, with too much parallelism you can overload the RDBMS so much that it leads to timeout errors, which cause processing to fail. By default, the maximum number of concurrent connections, and thus queries, for a data source is limited to ten. This can be changed by altering the Maximum Number of Connections setting of the data source properties in either Business Intelligence Development Studio or SQL Server Management Studio.
To help you monitor the number of partitions processing in parallel, you can review the MSAS 2005:Processing\Rows read/sec performance counter. Generally you should expect this counter to display 40,000–60,000 rows per second for one partition. If your partition contains complex SQL joins or hundreds of source columns, you are likely to see a lower rate. Additionally, you can monitor the number of threads being used during processing by using the MSAS 2005: Threads\Processing pool busy threads performance counter. You can also view jobs that are waiting to execute by using the MSAS 2005: Threads\Processing pool job queue length performance counter.
Note that when you perform parallel processing of any object, all parallel operations are committed in one transaction. In other words, it is not possible to perform a parallel execution and then commit each transaction as it progresses. While this is not specifically a performance issue, it does impact your processing progress. If you encounter any errors during processing, the entire transaction rolls back.
Use sufficient memory
The Optimizing memory usage section describes techniques to ensure that Analysis Services has sufficient memory to perform querying and processing operations. Ensuring that Analysis Services has sufficient memory can also impact Analysis Services usage of processor resources. If the Analysis Services server has sufficient memory, the Windows operating system will not need to page memory from disk. Paging reduces processing performance and query responsiveness.
Use a load-balancing cluster
If your performance bottleneck is processor utilization on a single system as a result of a multi-user query workload, you can increase query performance by using a cluster of Analysis Services servers to service query requests. Requests can be load balanced across two Analysis Services servers, or across a larger number of Analysis Services servers to support a large number of concurrent users (this is called a server farm). Load-balancing clusters generally scale linearly. Both Microsoft and third-party vendors provide cluster solutions. The Microsoft load-balancing solution is Network Load Balancing (NLB), which is a feature of the Windows Server operating system. With NLB, you can create an NLB cluster of Analysis Services servers running in multiple host mode. When an NLB cluster of Analysis Services servers is running in multiple host mode, incoming requests are load balanced among the Analysis Services servers. When you use a load-balancing cluster, be aware that the data caches on each of the servers in the load-balancing cluster will be different, resulting in differences in query response times from query to query by the same client.
A load-balancing cluster can also be used to ensure availability in the event that a single Analysis Services server fails. An additional option for increasing performance with a load-balancing cluster is to distribute processing tasks to an offline server. When new data has been processed on the offline server, you can update the Analysis Services servers in the load-balancing cluster by using Analysis Services database synchronization.
If your users submit a lot of queries that require fact data scans, a load-balancing cluster may be a good solution. For example, queries that may require a large number of fact data scans include wide queries (such as top count or medians), and random queries against very complex cubes where the probability of hitting an aggregation is very low.
However, a load-balancing cluster is generally not needed to increase Analysis Services performance if aggregations are being used to resolve most queries. In other words, concentrate on good aggregation and partitioning design first. In addition, a load-balancing cluster does not solve your performance problem if processing is the bottleneck or if you are trying to improve an individual query from a single user. Note that one restriction to using a load-balancing cluster is the inability to use writeback, because there is no single server to which to write back the data.
Understanding how Analysis Services uses disk resources
Analysis Services uses disk I/O resources for both querying and processing. Increasing the speed of your disks, spreading the I/O across multiple disks, and using multiple controllers, can significantly improve processing performance. These steps also significantly improve query responsiveness when Analysis Services is required to perform fact data scans. If you have a large number of queries that require fact data scans, Analysis Services can become constrained by insufficient disk I/O when there is not enough memory to support the file system cache in addition to Analysis Services memory usage.
Disk resource demands during processing
As stated previously in the Memory demands during processing section, during processing, the aggregation buffer determines the amount of memory that is available to build aggregations for a given partition. If the aggregation buffer is too small, Analysis Services uses temporary files. Temporary files are created in the TempDir folder when memory is filled and data is sorted and written to disk. When all necessary files are created, they are merged together to the final destination. Using temporary files can result in some performance degradation during processing; however, the impact is generally not significant given that the operation is simply an external disk sort. Note that this behavior is different than previous versions of Analysis Services. To monitor any temporary files used during processing, review the MSAS 2005:Proc Aggregations\Temp file bytes written/sec or the MSAS 2005:Proc Aggregations\Temp file rows written/sec performance counters.
Disk resource demands during querying
During querying, Analysis Services may request arbitrary parts of the data set, depending on user query patterns. When scanning a single partition, the I/Os are essentially sequential, except that large chunks may be skipped because the indexes may indicate that they aren’t needed. If commonly used portions of the cube (particularly the mapping files) fit in the file system cache, the Windows operating system may satisfy the I/O requests from memory rather than generating physical I/O. With large cubes, using a 64-bit version of the Microsoft Windows Server 2003 family increases the amount of memory that the operating system can use to cache Analysis Services requests. With sufficient memory, much of the cube can be stored in the file system cache.
Optimizing disk usage
While increasing disk I/O capacity can significantly improve the overall performance of Analysis Services, there are several steps you can take to use existing disk I/O more effectively. This section contains guidelines to help you optimize disk usage of Analysis Services.
Using sufficient memory
The Optimizing memory usage section describes techniques to ensure that Analysis Services has sufficient memory to perform querying and processing operations. Ensuring that Analysis Services has sufficient memory can also impact Analysis Services usage of disk resources. For example, if there is not enough memory to complete processing operations, Analysis Services uses temporary files, generating disk I/O.
If you cannot add sufficient physical memory to avoid memory paging, consider creating multiple paging files on different drives to spread disk I/O across multiple drives when memory paging is required.
Optimizing file locations
The following techniques can help you to optimize the data files and temporary files used during processing:
• Place the Analysis Services data files on a fast disk subsystem.
The location of the data files is determined by the DataDir server property. To optimize disk access for querying and processing, place the Analysis Services Data folder on a dedicated disk subsystem (RAID 5, RAID 1+0, or RAID 0+1).
• If temporary files are used during processing, optimize temporary file disk I/O.
The default location of the temporary files created during aggregation processing is controlled by the TempDir property. If a temporary file is used, you can increase processing performance by placing this temporary folder on a fast disk subsystem (such as RAID 0 or RAID 1+0) that is separate from the data disk.
Disabling unnecessary logging
Flight Recorder provides a mechanism to record Analysis Services server activity into a short-term log. Flight Recorder provides a great deal of benefit when you are trying to troubleshoot specific querying and processing problems; however, it introduces a certain amount of I/O overheard. If you are in a production environment and you do not require Flight Recorder capabilities, you can disable its logging and remove the I/O overhead. The server property that controls whether Flight Recorder is enabled is the Log\Flight Recorder\Enabled property. By default, this property is set to true.
Conclusion
For more information:
http://www.microsoft.com/technet/prodtechnol/sql/2005/technologies/ssasvcs.mspx
Did this paper help you? Please give us your feedback. On a scale of 1 (poor) to 5 (excellent), how would you rate this paper?
Appendix A – For More Information
The following white papers might be of interest.
- Analysis Services 2005 Migration
- Real Time Business Intelligence with Analysis Services 2005
- Strategies for Partitioning Relational Data Warehouses in Microsoft SQL Server
- Analysis Services 2005 Processing Architecture
- Introduction to MDX Scripting in Microsoft SQL Server 2005
- Project REAL Monitoring and Instrumentation
- Project REAL: Analysis Services Technical Drilldown
- Project REAL: Enterprise Class Hardware Tuning for Microsoft Analysis Services
Appendix B – Partition Storage Modes
Each Analysis Services partition can be assigned a different storage mode that specifies where fact data and aggregations are stored. This appendix describes the various storage modes that Analysis Services provides: multidimensional OLAP (termed MOLAP), hybrid OLAP (HOLAP), and relational OLAP (ROLAP). Generally speaking. MOLAP provides the fastest query performance; however, it typically involves some degree of data latency.
In scenarios where you require near real-time data refreshes and the superior query performance of MOLAP, Analysis Services provides proactive caching. Proactive caching is an advanced feature that requires a special set of performance tuning techniques to ensure that it is applied effectively. For more information on the performance considerations of using proactive caching, see Near real-time data refreshes in this white paper.
Multidimensional OLAP (MOLAP)
MOLAP partitions store aggregations and a copy of the source data (fact and dimension data) in a multidimensional structure on the Analysis Services server. All partitions are stored on the Analysis Services server.
Analysis Services responds to queries faster with MOLAP than with any other storage mode for the following reasons:
- Compression—Analysis Services compresses the source fact data and its aggregations to approximately 30 percent of the size of the same data stored in a relational database. The actual compression ratio varies based on a variety of factors, such as the number of duplicate keys and bit encoding algorithms. This reduction in storage size enables Analysis Services to resolve a query against fact data or aggregations stored in a MOLAP structure much faster than against data and aggregations stored in a relational structure because the size of the physical data being retrieved from the hard disk is smaller.
- Multidimensional data structures—Analysis Services uses native multidimensional data structures to quickly find the fact data or aggregations. With ROLAP and HOLAP partitions, Analysis Services relies on the relational engine to perform potentially large table joins against fact data stored in the relational database to resolve some or all queries. Large table joins against relational structures take longer to resolve than similar queries against the MOLAP structures.
- Data in a single service—MOLAP partitions are generally stored on a single Analysis Services server, with the relational database frequently stored on a server separate from the Analysis Services server. When the relational database is stored on a separate server and partitions are stored using ROLAP or HOLAP, Analysis Services must query across the network whenever it needs to access the relational tables to resolve a query. The impact of querying across the network depends on the performance characteristics of the network itself. Even when the relational database is placed on the same server as Analysis Services, inter-process calls and the associated context switching are required to retrieve relational data. With a MOLAP partition, calls to the relational database, whether local or over the network, do not occur during querying.
Hybrid OLAP (HOLAP)
HOLAP partitions store aggregations in a multidimensional structure on the Analysis Services server, but leave fact data in the original relational database. As a result, whenever Analysis Services needs to resolve a query against fact data stored in a HOLAP partition, Analysis Services must query the relational database directly rather than querying a multidimensional structure stored on the Analysis Services server. Furthermore, Analysis Services must rely on the relational engine to execute these queries. Querying the relational database is slower than querying a MOLAP partition because of the large table joins generally required.
Some administrators choose HOLAP because HOLAP appears to require less total storage space while yielding excellent query performance for many queries. However, these apparent justifications for using HOLAP storage option are negated by the likelihood of excessive aggregations and additional indexes on relational tables.
- Excessive aggregations—Query responsiveness with HOLAP partitions relies on the existence of appropriate aggregations so that Analysis Services does not have to resolve queries against the fact table in the relational database. To ensure that a wide range of aggregations exists, administrators sometimes resort to generating excessive aggregations by increasing the performance improvement percentage in the Aggregation Design Wizard, or artificially increasing the partition row counts (and sometimes both). While these techniques increase the percentage of queries that Analysis Services can resolve using aggregations, there will always be some queries that can only be resolved against the fact data (remember the one-third rule). In addition, generating additional aggregations to improve query responsiveness comes at the cost of significantly longer processing times and increased storage requirements (which also negates the space savings).
- Additional indexes on relational tables—To ensure that the relational engine can quickly resolve queries that Analysis Services must resolve against the fact table in the relational database, administrators often add appropriate indexes to the fact and dimension tables. These additional indexes frequently require more space than MOLAP requires to store the entire cube. The addition of these indexes negates the apparent savings in disk space that is sometimes used to justify HOLAP. In addition, maintaining the indexes on the relational tables slows the relational engine when adding new data to the relational tables.
From a processing perspective, there is no significant difference in processing performance between MOLAP partitions and HOLAP partitions. In both cases, all fact data is read from the relational database, and aggregations are calculated. With MOLAP, Analysis Services writes the fact data into the MOLAP structure. With HOLAP, Analysis Services does not store fact data. This difference has minimal impact on processing performance, but can have a significant impact on query performance. Because HOLAP and MOLAP processing speeds are approximately the same and MOLAP query performance is superior, MOLAP is the optimum storage choice.
Relational OLAP (ROLAP)
ROLAP partitions store aggregations in the same relational database that stores the fact data. By default, ROLAP partitions store dimensions in MOLAP on the Analysis Services server, although the dimensions can also be stored using ROLAP in the relational database (for very large dimensions). Analysis Services must rely on the relational engine to resolve all queries against the relational tables, storing both fact data and aggregations. The sheer number of queries with large table joins in large or complex cubes frequently overwhelms the relational engine.
Given the slower query performance of ROLAP, the only situation in which ROLAP storage should be used is when you require reduced data latency and you cannot use proactive caching. For more information on proactive caching, see Near real-time data refreshes in this white paper. In this case, to minimize the performance cost with ROLAP, consider creating a small near real-time ROLAP partition and create all other partitions using MOLAP. Using MOLAP for the majority of the partitions in a near real-time OLAP solution allows you to optimize the query responsiveness of Analysis Services for most queries, while obtaining the benefits of real-time OLAP.
From a processing perspective, Analysis Services can store data, create MOLAP files, and calculate aggregations faster than a relational engine can create indexes and calculate aggregations. The primary reason the relational engine is slower is due to the large table joins that the relational engine must perform during the processing of a ROLAP partition. In addition, because the relational engine performs the actual processing tasks, competing demands for resources on the computer hosting the relational tables can negatively affect processing performance for a ROLAP partition.
Appendix C – Aggregation Utility
As a part of the Analysis Services 2005 Service Pack 2 samples, the Aggregation Utility is an advanced tool that complements the Aggregation Design Wizard and the Usage-Based Optimization Wizard by allowing you to create custom aggregation designs without using the aggregation design algorithm. This is useful in scenarios where you need to override the algorithm and create a specific set of aggregations to tune your query workload. Rather than relying on the cost/benefit analysis performed by the algorithm, you must decide which aggregations are going to be most effective to improve query performance without negatively impacting processing times.
Benefits of the Aggregation Utility
The Aggregation Utility enables you to complete the following tasks.
View and modify specific aggregations in an existing aggregation design.
Using the Aggregation Utility, you can view, add, delete, and change individual aggregations in existing designs. Once you build an aggregation design using the Aggregation Design Wizard or Usage-Based Optimization Wizard, you can use the utility to view the attributes that make up each aggregation. In addition, you have the ability to modify an individual aggregation by changing the attributes that participate in the aggregation.
Create new aggregation designs.
You can either create new aggregation designs by manually selecting the attributes for the aggregations, or by using the utility to build aggregations based on the query log. Note that the Aggregation Utility’s ability to build aggregations from the query log is very different than the functionality of the Usage-Based Optimization Wizard. Remember that the Usage-Based Optimization Wizard reads data from the query log and then uses the aggregation design algorithm to determine whether or not an aggregation should be built. While the Usage-Based Optimization Wizard gives greater consideration to the attributes contained in the query log, there is no absolute guarantee that they will be built.
When you use the Aggregation Utility to build new aggregations from the query log, you decide which aggregations provide the most benefit for your query performance without negatively impacting processing times. In other words, you are no longer relying on the aggregation design algorithm to select which aggregations are built. To help you make effective decisions, the utility enables you to optimize your design, including the ability to remove redundancy, eliminate duplicates, and remove large aggregations that are close to the size of the fact table.
Review whether aggregations are flexible or rigid.
A bonus of the Aggregation Utility is the ability to easily identify whether an aggregation is flexible or rigid. By default, aggregations are flexible. Remember that in a flexible aggregation, one or more attributes have flexible relationships while in a rigid aggregation, all attributes have rigid relationships. If you want change an aggregation from flexible to rigid, you must first change all of the necessary attribute relationships. Once you make these changes, you can use the utility to confirm that you have been successful as the aggregation will now be identified as rigid. Without the utility, you need to manually review the aggregation files in the operating system to determine whether they were flexible or rigid. For more information on rigid and flexible aggregations, see Evaluating rigid vs. flexible aggregations in this white paper.
How the Aggregation Utility organizes partitions
Using the Aggregation Utility, you can connect to an instance of Analysis Services and manage aggregation designs across all of the cubes and databases in that instance. For each measure group, the Aggregation Utility groups partitions by their aggregation designs. Figure 32 displays an example of this grouping.
Figure 32 Aggregation display for the Internet Sales measure group
The partitions in Figure 32 are grouped as follows:
- Aggregation design created by a wizard—Aggregation designs created by the Aggregation Design Wizard or the Usage-Based Optimization Wizard are automatically named AggregationDesign, with an optional number suffix if more than one Wizard-created aggregation design exists per measure group. For example, in the Internet Sales measure group, the Internet_Sales_2002 partition has an aggregation design named AggregationDesign, and the Internet_Sales_2001 partition contains an aggregation design named AggregationDesign 1. Both AggregationDesign and AggregationDesign 1 were created by one of the wizards.
- Aggregation design created by the Aggregation Utility—The Internet_Sales_2003 and Internet_Sales_2004 partitions share an aggregation design, called AggregationUtilityExample. This aggregation design was created by the Aggregation Utility. The Aggregation Utility allows you to provide a custom name for each aggregation design.
- No Aggregation Design—For the Internet Orders measure group, none of the partitions in that measure group have an aggregation design yet.
How the Aggregation Utility works
The most common scenario for using the Aggregation Utility is to design aggregations based on a query log. Following is a list of steps to effectively use the Aggregation Utility to design new aggregations based on the query log.
To add new aggregations based on the query log
-
Perform pre-requisite set up tasks.
Before using the Aggregation Utility, you must configure Analysis Services query logging just as you would before you use the Usage-Based Optimization Wizard. As you set up query logging, pay close attention to configure an appropriate value for the QueryLogSampling property. The default value of this property is set to one out of every ten queries. Depending on your query workload, you may need to increase this value in order to collect a representative set of queries in the Analysis Services query log table. Obtaining a good sampling of queries is critical to effectively using the Aggregation Utility.
-
Add a new aggregation design based on a query log.
To extract data from the query log table, the Aggregation Utility provides a default query that returns a distinct list of datasets for a given partition. A dataset is the subcube that is used to satisfy query requests. An example of the default query is depicted in the query below. The values highlighted in yellow are placeholder values.
Select distinct dataset from OLAPQueryLog
Where MSOLAP_Database = DatabaseName and
MSOLAP_ObjectPath = MeasureGroupName
Generally speaking, it is a good idea to modify the default SQL statement to apply additional filters that restrict the records based on Duration or MSOLAP_User. For example, you may only return queries where the Duration > 30 seconds or MSOLAP_User = Joe.
In addition, whenever you add a new aggregation design by using the Aggregation Utility, it is a good idea to use a special naming convention to name the aggregation design as well as the aggregation prefix. This allows you to easily identify those aggregations that have been created by the utility. For example, when you use SQL Server Profiler to analyze the effectiveness of your aggregations, with easily recognizable names, you will be able to quickly identify those aggregations that have been created by the Aggregation Utility.
-
Eliminate redundant aggregations.
You can optimize a new aggregation design by eliminating redundant aggregations. Redundant aggregations are those aggregations that include one or more attributes in the same attribute relationship tree.
The aggregation highlighted in Figure 33 identifies an aggregation with attributes from two dimensions: Product and Time. From the product dimension, the aggregation includes the English Product Category Name
attribute. From the Time dimension, the aggregation includes the following attributes: English Month Name, Calendar Quarter, and Calendar Year. This is a redundant aggregation since English Month Name, Calendar Quarter, and Calendar Year are in the same attribute relationship tree.
Figure 33 Redundant aggregation example
To remove the redundancy in this aggregation, use the Eliminate Redundancy option in the Aggregation Utility. Figure 34 displays the aggregation after the Eliminate Redundancy option is applied. The aggregation now only includes the English Month Name attribute from the Time dimension.
Figure 34 Aggregations with redundancy eliminated
-
Eliminate duplicate aggregations.
Duplicate aggregations are aggregations that include the exact same set of attributes. Continuing with example in Figure 34, note that the there are two identical aggregations for 0000000,010000,0100. This aggregation consists of the English Product Category
Name and English Month Name attributes. After the Eliminate Duplicates option is applied, the duplicated aggregation is removed and the updated aggregation design is presented in Figure 35.
Figure 35 Aggregations with duplicates eliminated
-
Assign the aggregation design to a partition.
After you assign the aggregation design to one or more partitions, the utility displays the assigned partitions under the name of the new aggregation design, as displayed in Figure 32.
-
Save the measure group to SQL Server.
Your new aggregation design and partition assignment is not saved on the server until you perform an explicit save on the modified measure group. If you exit out of the utility and do not save, your changes will not be committed to the server.
-
Process the
partition.
Process the necessary measure group or partitions to build the aggregations for your new design. This operation needs to be performed outside of the Aggregation Utility using your normal processing techniques. Note that if you simply need to build aggregations, you can perform a ProcessIndexes operation on the appropriate measure group / partitions. For more information on ProcessIndexes, see Partition-processing commands.
-
Evaluate
the
aggregation size.
With the aggregations processed, you can use the Aggregation Utility to evaluate the relative size of each aggregation compared with the fact data for the partition. Using this information, you manually eliminate relatively large aggregations that take a long time to process and do not offer significant querying benefits. Remember that the aggregation design algorithm eliminates any aggregations that are greater than one third the size of the fact table. To apply similar logic, you can easily identify and delete any aggregations in your custom aggregation design that are significantly large.
-
Re-save to SQL Server and reprocess.
After you evaluate the aggregation size and remove any large aggregations, re-save the aggregation design and then reprocess the necessary partitions. For any subsequent changes that you make over time, always remember to re-save and reprocess.
UPGRADING FARMS FROM SHAREPOINT 2007 TO SP2010
INTRODUCTION ………………………………………………………………………………………………………….. 4 1.1. Outline ……………………………………………………………………………………………………………………… 4 1.2. Acknowledgements ………………………………………………………………………………………………….. 4 1.3. Updates ……………………………………………………………………………………………………………………. 4 2. UPGRADING FARMS FROM SHAREPOINT 2007 TO SP2010 ……………………………………………….. 5 2.1. The upgrade cycle ……………………………………………………………………………………………………. 5 2.1.1. Learn …………………………………………………………………………………………………………………….. 6 Requirements and prerequisites ………………………………………………………………………………….. 6 Upgrade methods …………………………………………………………………………………………………….. 9 Downtime mitigation processes ………………………………………………………………………………… 16 2.1.2. Prepare ……………………………………………………………………………………………………………….. 19 Document environment …………………………………………………………………………………………… 19 Manage customizations …………………………………………………………………………………………… 19 Choose upgrade strategy ………………………………………………………………………………………… 24 2.1.3. Test ……………………………………………………………………………………………………………………… 24 Build test farms ………………………………………………………………………………………………………… 25 Document and install customizations ………………………………………………………………………… 25 Use real data…………………………………………………………………………………………………………… 25 Evaluate techniques ………………………………………………………………………………………………… 25 Find issues early ……………………………………………………………………………………………………….. 26 2.1.4. Implement …………………………………………………………………………………………………………… 27 Build/upgrade farms ………………………………………………………………………………………………… 27 Deploy customizations ……………………………………………………………………………………………… 27 Minimize downtime ………………………………………………………………………………………………….. 27 Monitor progress ……………………………………………………………………………………………………… 27 2.1.5. Validate ………………………………………………………………………………………………………………. 28 Upgrade event failures …………………………………………………………………………………………….. 28 UI/UX issues ……………………………………………………………………………………………………………… 28 Data issues ……………………………………………………………………………………………………………… 28 2.2. Visual upgrade ………………………………………………………………………………………………………… 28 2.3. No International Domain Name support…………………………………………………………………….. 28 3. UPGRADING SOLUTIONS AND CODE ………………………………………………………………………….. 29 3.1. Recompilation…………………………………………………………………………………………………………. 29 3.2. Upgrading Custom Site Definitions …………………………………………………………………………….. 29 3.2.1. Upgrade definition files …………………………………………………………………………………………. 30 3.3. Upgrading Solutions …………………………………………………………………………………………………. 30 3.4. Versioned Features ………………………………………………………………………………………………….. 31 3.4.1. Declarative feature upgrade ………………………………………………………………………………… 31 3.4.2. Programmatic feature upgrade …………………………………………………………………………….. 32 3.5. Customizations against deprecated/changed UI……………………………………………………….. 32 3.6. Security changes …………………………………………………………………………………………………….. 33 3.6.1. Web Parts …………………………………………………………………………………………………………….. 33 3.6.2. Sandboxed Solutions …………………………………………………………………………………………….. 33 3.7. Large List Throttling …………………………………………………………………………………………………… 34 3.8. Deprecated API’s ……………………………………………………………………………………………………. 35 3.9. Hardcoding issues ……………………………………………………………………………………………………. 35 3.10. Upgrading the look & feel to the new version ………………………………………………………….. 35 3.11. Upgrading projects to Visual Studio 2010 ………………………………………………………………… 36 3.12. Client upgrades ……………………………………………………………………………………………………. 38 4. PLANNING ………………………………………………………………………………………………………………. 38 4.1. Planning prerequisites ………………………………………………………………………………………………. 38 4.2. Planning upgrade model …………………………………………………………………………………………. 39 4.3. Planning new Server Architecture ……………………………………………………………………………… 39 4.4. Test, test, test …………………………………………………………………………………………………………… 40 4.5. Planning operations scheduling ………………………………………………………………………………… 41 4.6. Planning code upgrade approach …………………………………………………………………………… 41 4.7. Planning user adoption…………………………………………………………………………………………….. 42 1. INTRODUCTION 1.1. Outline This document describes guidance for upgrading a SharePoint Products and Technologies 2007 (SP2007) farm to SharePoint Products and Technologies 2010 (SP2010). The various approaches to upgrade will be described and the pros and cons of each approach will be considered. The first part of this document will discuss the process of performing an upgrade from SP2007 to SP2010 to include preparation, methodology and finalization. The second part of this document will focus on the upgrading custom solutions and will discuss some of the various tools and features available to assist in this process. The final part will look at actions that should be started, to prepare a solution for an upgrade. I have tried to encompass both the operations and development angle of the upgrade process in this white paper. This have sometimes forced me to not dig as much into a given subject as I probably would have liked to do, in an attempt to get this document finished.. 1.2. Acknowledgements A few people helped me by reading the initial drafts and suggesting changes: Mike Watson (http://www.sharepointmadscientist.com), Paul Swider (http://www.paulswider.com) and Wictor Wilén (http://wictorwilen.se). I know you guys are busy, so thanks a lot for some great input! In my research for this white paper I have read a lot of blogs and specs, and watched a lot of screen casts on the subject. I have tried to give credit where credit is due, but should I have missed accreditation let me know and I will include it. I will also appreciate any feedback and corrections from the ever growing SharePoint community. Note: This document should be considered a work-in-progress. As very few actual upgrades has been carried out at this point in time (SharePoint 2010 still being in beta) prescriptive guidance is scarce. It is my plan to keep this white paper up to date as best practices become established. 1.3. Updates Date Changed 5/9/2010 Chapter 3.1 – 3.3 updated with more info Added IDN upgrade and BDC upgrade 2. UPGRADING FARMS FROM SHAREPOINT 2007 TO SP2010 Note: There is no upgrade path from the public beta version of SP2010 to the RTM when released. Beta can be used to evaluate the product and to test upgrades, but since it is not a supported product, upgrade is not supported! 2.1. The upgrade cycle When talking upgrade of complex SharePoint solutions, it is important to emphasize that this initially is an iterative approach: Learn •find out all about requirements, prerequisites, documentation, the upgrade process, downtime mitigation, common issues Prepare •document environment thoroughly, upgrade existing documentation, find and manage customizations, choose upgrade strategy, performance test existing hardware Test •build a test farm using real data, evaluate migration techniques, find issues early Implement •upgrade farms, deploy customizations, minimize downtime, monitor progress Validate •upgrade event failures, UI/UX issues, data issues 2.1.1. Learn Requirements and prerequisites Software and hardware The biggest change in architecture from 2007 to SP2010 is, that all servers, including SQL server, must run 64-bit. This is mainly because of scalability issues, the need for large amounts of RAM on the server and to focus support on one version. The minimal requirements for hardware is pt specified to be: Component Minimum requirement Processor 64-bit, four cores RAM 4 GB for developer or evaluation use 8 GB for single server and multiple server farm installation for production use Hard disk 80 GB for system drive For production use, you need additional free disk space for day-to-day operations. Maintain twice as much free space as you have RAM for production environments. For more information, see Capacity management and sizing for SharePoint Server 2010. Table 1: Source TechNet Note: Now TechNet actually has a whole Capacity Management Resource Center for SharePoint 2010 dedicated to capacity planning and performance here http://technet.microsoft.com/enus/ sharepoint/ff601870.aspx In addition to hardware requirements for running SP2010, one must also consider the upgrade process itself and how it may be impacted by hardware as well. For example, the upgrade process may take 4 hours on one set of hardware and 2 hours on another. The speed of the upgrade will be determined in large part by the physical resources available to the SQL server(s). Expect the upgrade to run much faster when the SQL server(s) performing the upgrade has excess processor, memory and physical disk IO capacity. Also, the upgrade process can benefit greatly by scaling out across multiple SQL instances with each instance running one or more upgrade processes. Also be aware that the upgrade itself takes up extra disk space for log files and databases. Also you need to be running Windows Server 2008 R2 or Windows Server 2008 with SP2 on all servers (see this article for upgrade process http://technet.microsoft.com/enus/ library/cc288690.aspx). Furthermore database server must be 64-bit version of either SQL Server 2005 SP3 with cumulative update 3 (CU) or SQL Server 2008 SP1 with cumulative update 2. SharePoint 2007 must have SP2 and latest CU (currently April CU), since a lot of the tools used for upgrading SharePoint is baked into the service packs. Note: The above prerequisite upgrades can be combined, but must not be part of the SharePoint upgrade itself! Read more on hardware and software requirements on TechNet http://technet.microsoft.com/en-us/library/cc262485(office.14).aspx Pre-upgrade check That the tools are already in place also means that you can start planning an upgrade, by running the pre-upgrade checker. The stsadm.exe command PreUpgradeCheck can be used to analyze the existing SP2007 site collections, looking for situations that could cause grief during an upgrade, such as customized (unghosted) artifacts, changes in database schemas, missing features and other potential issues and relevant information like Alternate Access Mappings (AAM) url’s, site definitions used and large lists. It is also important to state, that the tools operations on the databases are read-only! No changes are made on the databases, which makes it a relatively harmless procedure to run, even on your production environment (in comparison with PreScan from 2003-2007 upgrades that would make small alterations to the databases). During execution PreUpgradeCheck will visually display progress in the console. • Green text means everything is fine • Yellow means you will find more information available when digging into the log file with references to KB articles; manual upgrades will also show up here. Below you can see that CAML views are used instead of the new XSLT-based views, this needs to be upgraded manually, also listed are AAM configuration, server and farm info, installed language packs etc. • Red means that there’s an issue that needs attention before an upgrade can be completed successfully. In the example below the upgrade fails to find the xml for an installed feature and also fails on the prerequisites (the server is 32-bit). Figure 1: Pre-Upgrade Check in action (source: http://www.wictorwilen.se) After PreUpgradeCheck has finished, it will generate a report in both XML and HTML format and a log file. The PreUpgradeCheck runs against a rules database that is extendable. You can select what rules to run by specifying the rulefiles parameter followed by a commaseparated list of rule names. You can also see a list of rules being applied by specifying the listrulefiles parameter. Note: For a detailed walk-through of the reports generated see TechNet article http://technet.microsoft.com/en-us/library/cc262231(office.14).aspx and Joel Olsen’s blog http://www.sharepointjoel.com/Lists/Posts/Post.aspx?ID=238 PreUpgradeCheck can be run on both a single server and a whole farm. There are two obvious benefits of this: running it locally will only stress a single Web Frontend (WFE) server, which is good if run on a production environment. Also you can run PreUpgradeCheck on individual WFE and afterwards compare the reports against each other to spot inconsistencies across the frontend servers. PreUpgradeCheck is meant to be run a number of times, not just as a one-off event. Identifying customizations and rehearsing upgrade operation is paramount when we want to achieve a successful upgrade with minimal downtime. For this reason IT should run PreUpgradeCheck on a regular basis as an ongoing process towards the Verson-To-Version (V2V) upgrade. Common issues when upgrading is upgrading language packs to latest version, upgrading custom site definitions to take advantage of new SP2010 functionality (for more information, see http://tinyurl.com/mulfcb), missing features (only guid is stored in the database), large lists (SP2010 uses throttling on large lists, so code may fail! See more in code upgrade section) or orphaned artefacts in configuration or content database. Test-SPContentDatabase To complement PreUpgradeCheck reports, as part of the pre-upgrade testing you should run the SP2010 PowerShell (PS) command Test-SPContentDatabase. This command compares a content database and a web application against each other checking for problems. It can be used against both old 2007 content database and the upgraded SP2010 database. The tool will check for orphans, missing site definitions, features, assemblies etc. In other words it will warn you if it detects any potential problems with matching a specific web application and database, such as creating orphans by adding a database that is already in the farm. Figure 2: Test-SPContentDatabase example output So where Pre-upgrade Check is used to detect issues on the SP2007 environment, Test- SPContentDatabase can be used to for example analyze a SP2010 farm before attaching a content database to it. Upgrade methods Part of the learning process is knowing your options! There’s several ways to upgrade your SharePoint solution and even hybrid variations. Each method has its pros and cons, with concern for downtime, hardware costs etc. In-Place upgrade An in-place upgrade means that the upgrade is done directly on the production server. Since this means closing down the farm for the duration of the upgrade, this approach causes downtime for the solution. On the other hand the approach means that the existing server hardware can be reused (if within specifications and adhering to prerequisites) and that configurations and customizations done on the server is kept. E.g. you don’t need to recreate a complete farm using solutions and manual configuration. Figure 3: In-place upgrade (source TechNet) Doing an in-place upgrade, first install SP2010 on *all* servers in farm -start with the server hosting the Central Administration (CA). Then install language packs. Now run configuration wizard up to point where wizard tells you to configure other servers in farm -start with CA. When wizard is on same step on all servers complete wizard on CA continue on other servers. As an option you can end up running Visual Upgrade (the new SP2010 look for editing sites, with Ribbons etc.). If you have problems during in-place upgrade, the PS command Upgrade-SPContentDatabase can be used to resume an upgrade. Note: More information on In-place upgrade on TechNet: http://technet.microsoft.com/enus/ library/cc303423(office.14).aspx Pros: • customizations are kept • farm-wide settings preserved Cons: • a risky approach since you don’t have a fallback strategy should issues arise • downtime while upgrading (can be mitigated with AAM redirects, see hybrid model below) • all content databases are upgraded in sequence causing more downtime • a power outage or disk space problem during upgrade could leave upgrade in an unsupported state Database attach The database attach approach requires you to create a new farm on new hardware. This farm is then configured, and customizations and artifacts are deployed. Now you backup your old farm, detach it taking it offline and attach it to the new farm (discard temporary content database in new farm). Attaching the new database could be done with either the PowerShell command Mount- SPContentDatabase –name <newdb> -WebApplication <url>, or use stsadm –o addcontentdb –url <url> -databasename <dbname> [-preserveolduserexperience true|false]. The last approach should be preferred if you want control over the UI upgrade (e.g. Ribbons), since it honors the version switch for UI, whereas the PS command forces the new UI (at least until RTM version). This method is also viable for SSP database and upgrade user profile information into the database, but you cannot upgrade search database by using this method. If you have problems during db attach upgrade that you need to address before continuing, the upgrade process is designed so that it can be resumed even in the event of power outage or if you run out of space during the upgrade process: run the PS command Upgrade- SPContentDatabase to resume an upgrade. Note: More information on db attach upgrade on TechNet: http://technet.microsoft.com/enus/ library/cc303436(office.14).aspx Pros: • can upgrade multiple content databases in parallel (less downtime) • you can use this method to consolidate multiple farms into one farm • you can upgrade hardware as well as software • you have an opportunity to clean out the old server and get a “fresh” install Cons: • server and farm settings are not upgraded (mitigation: scripted installs) • the settings of the target farm must exactly match the settings on the source farm • customizations are not upgraded (mitigation: solution deployment, scripted configurations with PowerShell) • copying databases over network takes time (plan this!) • requires direct access to SQL server Figure 4: DB attach upgrade (source: TechNet) Hybrid approach 1: Read-only databases Hybrid approaches gives you the possibility of combining different approaches when upgrading SP2010. One such approach is the R/O databases approach. Basically this is a db attach upgrade but with a downtime mitigation strategy where you continue to provide read-only access to content database during the upgrade. Start by setting up and configuring a new farm, then transfer customizations to new farm and test. Now set content databases to read only (directly in SQL) on original farm while upgrade in progress on new farm (since Sp2 SharePoint will detect that the database is read-only so that the UI respects this). Backup content database from original farm and perform database upgrade on the new farm in parallel. Optionally use AAM for long-running upgrades to redirect requests (see more on this approach later). Map sites from new farm to old farm while upgrade is in progress. Note: You can configure the READ_ONLY database availability option by using Transact-SQL. More about how to use the SET clause of the ALTER DATABASE statement: http://go.microsoft.com/fwlink/?LinkId=148362). Figure 5: Hybrid 1: Read-only database (source TechNet) Pros: • Existing farm can continue to run in read-only mode causing minimal downtime for end users • can upgrade multiple content databases in parallel (less downtime) • you can use this method to consolidate multiple farms into one farm • you can upgrade both software and hardware Cons: • server and farm settings are not upgraded (mitigation: scripted installs) • customizations are not upgraded (mitigation: solution deployment, scripted configurations with PowerShell) • copying databases over network takes time (plan this!) • requires direct access to SQL server Hybrid approach 2: Detach databases Another hybrid approach is a variation over the in-place upgrade: This approach combines the in-place upgrade’s ability to keep configurations and customizations while adding the parallel upgrade approach from db attach positively affecting downtime for the upgrade: Take the original farm off-line, detach content database from original farm, run in place upgrade on original farm servers in parallel, services and configuration databases. Then attach content databases to the original farm and upgrade content. Figure 6: Hybrid: Detach databases (source TechNet) Pros: • customizations are kept • farm-wide settings preserved • save time by upgrading multiple db’s at the same time Cons: • copying databases over network takes time (plan this!) • requires direct access to SQL server Hybrid approach 3: Detach databases (with temporary farm) This approach is very similar to the above hybrid scenario, but it introduces a new small farm that is used temporarily to store the content databases as they are being upgraded: Set up temporary small farm (both WFE and applications running on same hardware) running SP2010 and then take the original farm offline. Detach the content databases from the original farm and run an in-place upgrade on original farm. Now attach content databases to temp farm and upgrade content in parallel. Finally re-attach content databases to the original farm. Figure 7: Hybrid: Detach databases with temporary farm (source: TechNet) Pros: • Same as hybrid 2 approach above + • Reduce downtime since upgrade is carried out in parallel on temp farm Cons: • Same as hybrid 2 approach above + • New hardware needed for temp farm (could be some existing test server) AAM hybrid: detach databases The AAM hybrid should be seen as a last ditch operation, and is only viable for very specific situations, like if you cannot upgrade your farm over a weekend. The reason for this being that it is operationally fairly difficult to set up. It also isn’t perfect; since it has issues with links (different URL’s on new and old farm). Furthermore it gives you double work (e.g. governance of security, double hardware, double maintenance). The upgrade is related to what in the old version was called Gradual Upgrade (no longer supported). Basically db attach is used to upgrade content databases one at the time over a longer period. AAM is then used on the new farm to redirect users that request pages that haven’t yet been upgraded to the old farm (http://WSSold). Over time (could be weeks or even months) all content databases are upgraded one at the time. Compared to Gradual Update the granularity here is entire content databases, not site collections. When the databases are upgraded the old databases could be kept as read-only as a kind of post view upgrade to look at old content to compare with new. Further details available in TechNet White Paper: http://technet.microsoft.com/dadk/ library/ee720448(en-us,office.14).aspx Updating Services Services have been totally reworked in SP2010. There is no longer a Shared Services Provider (SSP) site, but instead you got the possibility to scale out the services to individual servers (through proxies) with individual databases. This flexibility is great in terms of scaling out, but adds complexity to upgrade scenarios. You really need to plan beforehand what services are in use in the farm, and where they should be placed after upgrading to SP2010. Also some services are split up into two separate services, where one is completely new. Depending on the upgrade approach manual work is needed to fully upgrade the service architecture. Another important design change from 2007 to SP2010 is that where some services was specific to Microsoft Office SharePoint Server (MOSS) -some even only in Enterprise edition, they now all reside inside Microsoft SharePoint Foundation (formerly Windows SharePoint Services (WSS)). This should cause solution architects to consider the new possibilities available for the customers’ farm, maybe even change existing solutions to make use of these new possibilities. Important: Even with in-place upgrades, not all configurations are kept after upgrade. These settings, such as timer job configurations, must be collected before upgrade and re-applied post-upgrade. Below is an illustration of SSP architecture before and after an upgrade: If you have a single SSP, all proxies for service applications are added to the default proxy group. The following diagrams show the changes to your farm that are made during in-place upgrade. Services infrastructure before upgrade: Figure 8: SSP before and after upgrade (Source: TechNet) Note: If you have multiple SSP’s, they will be upgraded together and after the upgrade you will have multiple proxy groups! Technical diagrams illustrating services in SP2010: http://technet.microsoft.com/enus/ library/cc263199(office.14).aspx Logical architecture components – Service applications: http://technet.microsoft.com/enus/ library/cc263121(office.14).aspx#section2 User Profiles User Profiles are now split up in two services: • User Profile Service • Managed Metadata Service (new in SP2010) If you run an in-place upgrade, the managed metadata service is automatically enabled and configured. If you upgrade using db attach you will need to enable and configure Managed metadata before upgrading! Persisted properties relating to profiles are also preserved when using in-place upgrades: • MySiteHostURL • SearchCenterURL • EnablePersonalFeaturesforMultipleDeployments • ProfileStoreLanguage • ProfileStoreLanguagePacksApplied • ProfileStoreCollationID • DaysWorthOfEventsToKeep On the other hand a db attach approach will not preserve these properties since they are stored in configuration database. You also will need to enable and configure the Managed Metadata service before you upgrade the User profile service to make taxonomy data part of the upgrade. If you have taxonomy data that needs to be migrated (if you planned meta data before upgrading), use the Move-SPProfileManagedMetadataProperty command in PS. Note: To upgrade and use taxonomy data, the User Profiles Service proxy and Managed Metadata Service proxy must be in the same proxy group. My Sites If you use My Sites, make sure you upgrade the My Site host at the same time as you upgrade the user profiles. Also make sure you upgrade My Sites host as part of the intranet migration process! When you upgrade My Site host it will automatically upgrade to the new look and feel of SP2010, so any customizations on personal and shared My Site pages will be lost! Note: You don’t need to upgrade all the My Sites themselves at the same time as doing the User Profile upgrade, just the host! Search You cannot use db attach to upgrade search data. Instead you should configure search in your new farm before or after the upgrade. If you use in-place upgrade, you should review and adjust search topology after upgrade to suit new recommendations and requirements. Forms Services / InfoPath For db attach approach you need to export XSN files and UDCX files before upgrading and import them into new farm after upgrade: • Export-SPInfoPathAdministrationFiles • Update-SPInfoPathAdminFileUrl to update links if url is different in new farm You cannot use in-place for FormsServices. Excel Services Excel Services is still a local service (it runs service in same farm that consumes it). If you upgrade Excel Services using in-place upgrade: configuration info stored in SSP is automatically moved from SSP db to configuration database. When using the db attach approach, you need to reconfigure Excel Services on the new farm. After upgrade (db attach and in-place), a new unattended service account must be provisioned for Secure Store Service. Business Data Catalog (BDC) When you do an in-place upgrade, data from SSP is moved to a new dedicated database and a new service application is created. BDC is not upgraded in a db attach upgrade process. Old BDC Connections are run using Application Registry Backwards compatible service. The interface for this is kept in the old SSP admin site. New development should not be done in Application Registry Service, as this service is only meant to be used for upgrading BDC from SP2007! Note: If no BDC services were available for the old solution, the SSP site can be deleted after upgrade! Consider moving the BDC profile pages to a new location, as these were hosted in the SSP web application. Single Sign-On (SSO) The SSO service is replaced with Secure Store Service in SP2010. Use the PS cmdlets below to upgrade application definitions: • Upgrade-SPSingleSignOnDatabase • Upgrade-SSOConnectionString • Upgrade-SecureStoreConnectionString • Upgrade-SecureStorePassphrase Notice that passwords are not upgraded, so these will need to be configured post-upgrade. Also you must manually set Secure Store Service the default SSO provider after the upgrade is done. Downtime mitigation processes Usually you would like to minimize downtime during an upgrade. Several parameters affect your downtime: The chosen upgrade model, server performance, size of farm and databases, how well you tested etc. There are different processes that you can use to minimize downtime. Give users read access during upgrade One way is setting the source database to read-only during an upgrade. This will enable end users to access their data without changing it (SharePoint detects the SQL lock on the database and enforce UI trimming accordingly). The users will then only detect downtime when the solution is switched to the new farm. Upgrading in parallel To minimize the time used to run the upgrade use parallel upgrades. You can do parallel database attach (number of parallel upgrades depends on hardware) and create multiple temporary farms to do in-place upgrade and db attach on. Content database attach with AAM redirection is another way to reduce downtime. Avoid surprises – test! More subtle approaches could be to optimize farm before upgrade, to avoid surprises during the production upgrade: make sure you follow recommendations from pre-upgrade checker, split large content databases into smaller ones, test (on real data) –the more you rehearse the upgrade process, and the more “real” the test environment and test data are, the more certain you will be on a successful upgrade. Common issues that is only found through testing includes missing dependencies (features not deployed to new farm, or missing on one or more WFE), UI change (CSS will break if you just upgrade to the new UI without upgrading CSS), lack of space (for example on SQL server, you should expect x2-x3 space –especially depending on # of document versions- increase during an upgrade), there’s almost always some manual post-upgrade configuration that depending on setup needs to be done (for example configuring additional settings on Forms Authentication providers for claims-based web application). Clean up before upgrade It is very hard to predict the amount of time an upgrade will take. Performance will vary on a lot depending on farm metrics: • # site collections • # webs • # lists • # document versions • Document versions size • # documents • # links • Overall DB size To mitigate the above, do general “spring cleaning” on your site collections: delete unused sites, lists and documents. Clean up in number of versions for documents. Split up large content databases. Note: Remember to backup your databases before cleaning up! STSADM.EXE has operations to automate part of this procedure Delete live site collection: stsadm -o DeleteSite -url <URL> [-deleteadaccounts {True | False}] [- gradualdelete] Delete orphaned site collection: stsadm -o DeleteSite -force [-gradualdelete] -siteid <site ID> -databasename <database name> -databaseserver <database server name> Delete live site: stsadm -o DeleteWeb -url <URL> Delete orphaned site: stsadm -o DeleteWeb -force -webid <Web ID> -databasename <database name> – databaseserver <database server name> Since the amount of versions directly affect the time it takes to upgrade, consider manually deleting old document versions, or create a tool to automate this task. Clean up unused templates, features and web parts. Again this is a manual process, but a custom tool could automate the process (for example listing all unused templates and giving you the option to delete them). Repair data issues: stsadm -o DatabaseRepair -url <url> -databasename <database name> [- deletecorruption] stsadm -o ForceDeleteList -url <url> stsadm -o VariationsFixupTool -url <source variation site url> [-scan] [- recurse] [-label] [-fix] [-spawn] [-showrunningjobs] Check and remove locks on site collections (when doing backups): stsadm -o getsitelock -url <url> stsadm -o setsitelock -url <url> -lock {none | noadditions | readonly | noaccess} Revise hardware and server settings Performance also varies based on hardware and software metrics such as (in order of importance): • SQL disk I/O per sec. • SQL DB to disk layout • SQL temp db optimizations (one per cpu) • SQL CPU & memory • WFE CPU & memory • Network bandwidth & latency Revising the hardware and configuring server software before upgrading will help bring down the amount of downtime for an upgrade. 2.1.2. Prepare Document environment If your environment is not documented, this is the time to do this! If it is documented, this is the time to revise your documentation to ensure its up to date! You should document hardware, software, customizations (see more below) and configurations. This will assist you in estimating the scope of the upgrade, and make disaster recovery after a failed upgrade much easier. Manage customizations Probably one of the most common reasons for a failed upgrade is not knowing the extent of customizations on your farm. Are all customizations done using solution deployment? Are manual special case customizations that cannot easily be solved using solution deployment documented? And are these special cases in sync across WFE? Note: An upgrade is an excellent time to enforce government policies. If “rogue” customizations is found this should be followed up with guidance on packaging artifacts in solutions, using features etc. To answer these questions, you have a number of tools to help you, but you will also have to dig in GAC, bin, 12 hive, Solutions store, Add/remove programs, etc. to get an overview of the customizations on the farm. Examples of customizations include custom site/list definitions, themes and changed CSS, master pages, page layouts, content types, custom web parts, custom web controls, event handlers, customized/un-ghosted pages, application pages, custom timer jobs, AAM’s etc. The following section will try to shed some light on how to identify customizations in your farm: Pre-upgrade check First of all run pre-upgrade check tool on both farm and individual servers (running on individual servers and then comparing reports will give you a hint of how similar your WFE are). Note: List of all WSS/MOSS Pre-Upgrade Check KB articles: http://support.microsoft.com/kb/960577 Pre-Upgrade check on TechNet: http://technet.microsoft.com/en-us/library/dd793607.aspx Joel Oleson has a good blog post on the subject http://www.sharepointjoel.com/Lists/Posts/Post.aspx?ID=238 Customized/Unghosted files Pre-upgrade does a good job checking for customizations, but does not detect files customized (unghosted) in SharePoint Designer (SPD). A tool like Gary Lapointes gl-enumunghostedfiles (part of stsadm extensions http://stsadm.blogspot.com/2009/02/downloads.html) can help identifying and reghosting these customizations. Test the content database In SP2010 there’s a new tool available that will help identifying missing customizations: the PS cmdlet Test-SPContentDatabase can detect problems before you attach a content database to a farm. You can see this cmdlet as a compliment to pre-upgrade checker report, plus it works on both SP2010 and 2007 databases, so it is very useful to point at an upgraded database to check if assemblies, site definitions or features are missing or if there are undetected orphans. It also will show metrics for table sizing on a content database, which can be useful for detecting content approaching the software boundaries of the product. Note: Joel Oleson walks through the syntax and uses of Test-SPContentDatabase on his blog: http://www.sharepointjoel.com/Lists/Posts/Post.aspx?ID=288 EnumAllWebs Another tool to determine impact of customizations is stsadm –o enumallwebs. This command can be used to list the ID and sitemap status for all site collections and sub-sites in a specified content database. Especially sitemap status (InSiteMap=”True|False”) is useful, as this tells you if a site collection is orphaned in the content database (this could happen if a content database has been attached to a web application that already contained a site collection with the same URL). An orphan can both be a site only registered in content database, or a site only registered in the configuration database. Such orphans will need to be handled before upgrading the database. Note: Deleting of orphaned sites can be done using stsadm –o deletesite. More info on TechNet and Joel Oleson’s blog http://www.sharepointjoel.com/Lists/Posts/Post.aspx?ID=291 Always remember to backup your content database before deleting any sites or site collections! Deployment Advisor from Quest Deployment Advisor (DA) developed by Quest Software Inc., is a new tool due for release soon: One of the main purposes of this tool is to give Operations a way to get a sanity check on a given SharePoint farm: have the server been configured in compliance with best practices in the field? Does WFE contain unique configurations or customizations? Is the farm ready for an upgrade? Answering these and other questions makes Operations able to assess risks for SharePoint farms in regards to hardware, patches, customizations, security and performance. DA scans the farm against an extendable rules engine that describes best practices for SharePoint within categories such as Performance, SP2010 upgrade, Availability, Search, Security, Supportability and areas such as Antivirus, Farm Configuration, IIS, Network, Server and SQL. In an upgrade scenario, you can use DA to compare WFE servers (one of the ideas behind the tool is for it to be “the WinDiff of SharePoint”) with regards to configuration, customization, patches etc. You can also look at the specific farm with regards to 2010 upgrade issues. Here it will tell you what critical issues that need to be resolved before an upgrade can take place, such as upgrading to a 64-bit architecture on both web servers and SQL servers: You can also examine the SP2010 upgrade readiness for a specific server: One very powerful feature of DA is its ability to compare servers to each other. This mind you is across metrics such as hardware, software, patch-level, files on server, services on server etc. This proves useful both if you want to compare different WFE in the same farm, but also if you want to prepare for an upgrade: Say you create a clean install of SP2007, fully patched and following best practices. Then you compare that to the server you want to upgrade. That gives you the possibility to detect if files like core.js or other “Microsoft owned” files has been customized on the server in question. You can even filter on basically anything (like %.js) to fine tune your comparison. Very neat! Figure 9: Comparing servers in Deployment Advisor showing Core.js is customized In general this comparison against a “best practice server” is also useful if you take over a farm and want to quickly get an overview on the general state of the server by comparing metrics like BuildVersion, patch-level etc. with your “golden” server. Manual inspection A manual inspection of your farm could include • checking in Visual Studio and Solution store if everything is packaged in solutions • any manual editing of web.config (note that this will need to be checked both in relation to differences in web.config on different WFE in the farm, and across environments (devtest, integration test, preprod, prod) • any manual xcopy operations. These manual steps should be documented and if possible mitigated with solution deployment. Places to check for customizations • _layouts, features, sitedefinitions • GAC • add/remove programs (3rd party) • timerjobs, event receivers • http handlers/modules/iis customizations Pre-upgrade check does detect database customizations, but other kind of modifications of Out-Of-The-Box files such as webtemp files, application pages etc. will not be picked up. A way of detecting these customizations is using the above mentioned Depolyment Advisor, Windif (or similar) to detect differences a) from files as they were OOTB (install a clean farm and compare) b) between WFE on the same farm c) between environments Also inspect code, looking for hacks that may cause problems. A good developer would always mark these special cases with some kind of code comment. Since STP files are no longer supported, look for these in your development environment. A way to upgrade STP files to WSP packages is by restoring them on a SP2007 site that is then in-place upgraded. After fixing any visual issues the template can be exported as a WSP package that can then either be used to create new sites from using the UI, or be exported to Visual Studio 2010 and be packaged for deployment. Both the export and import tools has a tendency to import too much, so count on using time cleaning up the solutions before they are ready for deployment. More on this in a later chapter on upgrading code. Other tools for detecting customizations SPDiag version 2 is good for farm insight such as AAM’s or finding deployed solutions using the SnapShot tool. Diagnostics tool is also handy for detecting any discrepancies regarding best practices on configuration of the farm (Part of SharePoint administration Toolkit 4.0 that can be downloaded here http://technet.microsoft.com/en-us/library/cc508987.aspx). WssAnalyzeFeatures. This tool verifies if the feature definition files for all installed features are available on the file system, if the features used on a site collection are installed on the server (download from MSDN Code here http://code.msdn.microsoft.com/WssAnalyzeFeatures). Bamboo SharePoint Analyzer can help you get an overview of your farm topology, installed patches on servers, solutions and features deployed etc. (available here http://community.bamboosolutions.com/media/p/7160.aspx) SharePoint Feature Administration and Clean Up Tool can help locating faulty features in your farm (available from Codeplex http://featureadmin.codeplex.com) Collect customizations When all customizations has been collected, create a list of customizations along with source, environment and action required to move customization (could also be not to move it, e.g. if it’s a SP 2007 specific customization). The list should also contain third party add-ins and assemblies. When collecting customizations try and asses weather this customization is still relevant on the new platform: 1. Keep the customization. Choose if customization can be ported to new platform without issues. 2. Replace or redo customization. Choose if customization has visual or functional issues on the new platform, but you want to keep the customization. 3. Discard customization. Choose this if customization is no longer relevant. The following table illustrates common customizations and recommendation for that customization. Customization type Recommendation Site templates (STP files) STP files are a deprecated feature in SharePoint Server 2010. New site templates in SharePoint Server 2010 are saved as WSP files (solution packages). A site that was provisioned by using a site template will be upgraded, but you will be unable to create new sites based on that template. If you want to be able to create new sites, you can create and deploy a solution package instead. Site definition Migrate sites to a supported, predefined site definition, then apply custom features by using solution deployment. You can also continue to use a custom site definition. You do not have to create a new site definition based on SharePoint Server 2010. However, if you must perform custom upgrade actions for the definition, you might have to create an upgrade definition file for that site definition. For more information, see Upgrade Definition Files (http://go.microsoft.com/fwlink/?LinkId=182339) on MSDN. Feature Evaluate, then redesign or redeploy if necessary. Workflows and server controls Depends on the solution. Contact the vendor to find out whether there is an updated solution. If a workflow is compatible with the new version, redeploy. Event handler Rewrite and redeploy as a feature. Managed paths (inclusions/exclusions) Re-create inclusions for a database attach upgrade. Exclusions are assumed and do not have to be re-created. Themes Because of the extensive changes to the UI, custom themes based on Office SharePoint Server 2007 will not work in SharePoint Server 2010. Use Visual Upgrade to continue to use the sites in the old user experience until you can create and apply a new theme based on SharePoint Server 2010. Toolbar actions Move to the ribbon (Fluent UI). Master pages and CSS files Rework to accommodate the new user experience. JavaScript Test to determine whether any actions are required. In some cases, you might have to adjust the scripts to work with the new page model. Verify that it works on an upgraded site, and in both Visual Upgrade modes. Search provider or security trimmer Test to determine whether any actions are required. Web Parts Test to determine whether any actions are required. You might have to adjust the Web Parts to work with strict XHMTL mode. If a Web Part is located on a page but not in a Web Part Zone (so that it is, basically, HTML code embedded directly in a page), it will not work if you revert the page to the default template. Services Test to determine whether any actions are required. Redesign or adjust code, as needed. Authentication providers Test to determine whether any actions are required. Redeploy the provider on a test farm and ensure that it works correctly with claims authentication. Table 1- Source: TechNet Note: On TechNet you will find a worksheet that will help you document setup and collect customizations: http://go.microsoft.com/fwlink/?LinkId=179928 Choose upgrade strategy When customizations are collected, it is time to plan what upgrade strategy should be chosen, and determine order of operations (what sites goes first? should sites be split up?). Note: Even SharePoint behind the scenes will set recovery model to Simple during an upgrade (applicable for beta 2 in-place upgrade), you should still expect your SQL server to require x2-x3 of its current space –especially if you have a lot of versions on your documents. This is in part caused by the fact that databases aren’t shrinked automatically after an upgrade for time saving reasons. A How-To will come out shortly on TechNet on how to detect databases that need shrinking. The strategy should include means to limit downtime, and document expected downtime, and describe actions for spring cleaning as described earlier. It should also include a rollback strategy and a plan for when an upgrade should be abandoned and recovery of the old farm should start, any hardware upgrades due to new requirements, or space requirements. Note: It’s a good idea to do a performance analysis on your server hardware so you know beforehand if you should upgrade. System requirements for upgrade http://technet.microsoft.com/enus/ library/cc263322(office.14).aspx 2.1.3. Test The importance of testing before, during and after an upgrade cannot be stressed enough! It is imperative for the success of an upgrade that we have a test environment that we trust to be similar to the one we are going to upgrade in production. There are so many things that can go wrong during an upgrade, that without proper testing you could end up with either a long downtime, a site that’s not properly upgraded (missing features) or worse. Build test farms When you build test farms it is important that the metrics of the farm is kept as close to the production farm as possible! Both with regard to hardware, software, configuration, customizations and content they should be kept similar. The more similar your test farm is to the real thing, the higher the probability of everything running smooth during the actual upgrade in production. For hardware for example, the space on the disks plays an important factor: you would like to discover any space related issues during testing rather than having to add more disks during production upgrade. If the test environment is virtual, it should also be kept as close to the real farm as possible. You should for example run SQL server and the farms on different virtual images. If your tests environment isn’t identical you should keep it as similar as possible to the original: if you have multiple servers for a role (like 5 WFE) you should have at least 2 servers with that role in your test setup! Document and install customizations Use the worksheet mentioned above to document and install customizations and configurations. Use real data When you test the upgrade process, keep your content as close to production data as possible. This approach will help you identify trouble areas and determine upgrade performance. For example issues may rise due to large lists that you would not find on test data. Note: You don’t necessarily have to have all content on your upgrade test environment at the same time. Say you have 60 content databases with terabytes of data; it could be hard to convince your IT department to give you that kind of storage for a test farm. Instead test the content databases one at the time –just make sure you tested them all before attempting a real upgrade! Evaluate techniques After choosing the upgrade method you should do a test upgrade. This is just a preliminary test to catch any problems during the upgrade, and to rehearse the actual process. After the upgrade evaluate how things went, improve your techniques and do it again. And again! Evaluating also means troubleshooting problems, hunting for errors and validation of the result. Review log files To review the results of an upgrade, there’s several log files of interest: • pre-upgrade checker log file (in 12/LOGS dir) • psconfig log file (in 14/LOGS dir) • upgrade log file (in 14/LOGS dir) o find most recent log and look for a given correlation id • upgrade error log file (in 14/LOGS dir) If you search for and find the phrase “Upgrade session finished successfully!” the upgrade was went well. If the above entry was not found, search for ERROR and WARNING in upgrade log: • ERROR indicates failures such as failing components and faulty database connections • WARNING indicates issues such as missing features or components. Warnings should not be ignored. They may not break your upgrade process, but warnings should be investigated so you know what the impact will be on your system. Review sites For individual WFE you can also try and run stsadm -o localupgradestatus to find out if sites were skipped. If this is the case, you should restart the upgrade process. The before mentioned PS cmdlet Test-SPContentDatabase can also be used after an upgrade to validate if the content database has issues. Verify that the sites actually work using a browser, do a search crawl of the site and verify the crawl log for issues. Note: Since security scope has changed for deploying custom code in SharePoint, all test reviews should be done with a user with as low privileges as possible Reviewing artifacts A non-exhaustive list of things to check for when validating an upgraded site includes: Web parts • extra or missing web parts • broken web part pages • do they render correct • are any pages still checked out Style and appearance • images display correctly • CSS showing appropriately • themes showing appropriately • js working correctly (check for script errors) Permissions • does the appropriate people and groups still have correct permission level Customized (unghosted) pages • are customizations still in place • should customizations still be there in upgraded farm Find issues early Finding issues early ensures a higher success rate for the upgrade -the earlier we detect the problems the better. If you have multiple environments (as you should!), you can also use finding issues early to not repeat the problems found in test, in the subsequent environments such as integration test, preprod and prod, learning and improving the upgrade along the way. Note: TechNet has a couple of articles regarding testing and trial upgrades http://technet.microsoft.com/en-us/library/ff382642(office.14).aspx 2.1.4. Implement Build/upgrade farms First upgrade all farms to support the prerequisites for upgrade: upgrade to 64-bit, upgrade server OS, upgrade SQL server to supported versions, SP and CU, upgrade SharePoint to supported version, SP and CU. The process of getting the servers in a supported state can be combined, as long as you don’t combine the prerequisite upgrade with the SP2010 upgrade. Also upgrade hardware and build test farms. Depending on the chosen upgrade model, upgrade the services and content databases accordingly. Configure all valid settings, such as timer jobs, as recorded earlier. Prefer scripted configurations over manual ones, to minimize human error and ensure consistency across platforms. This is the case both for OS installations and server installations. Deploy customizations Again depending on upgrade model, it might be necessary to deploy all or at least some customizations. Make sure this is done as solutions whenever possible to ensure a consistent deploy across WFE. Minimize downtime Make sure that the SQL server is up for the job. When upgrading to SP2010 SQL server quickly becomes a bottleneck, so make sure it has plenty of space and horsepower if you want to minimize the time it takes to do an upgrade. Also consider making the content database read-only on the existing environment, while you upgrade a copy of this database in the background. Since SP2 SharePoint will detect that the database is read-only and trims the UI accordingly. This feature was added specifically with upgrade scenarios in mind! If you are doing db-attach upgrade, upgrading content databases in parallel will reduce the time it takes to upgrade. It is also possible to upgrade in parallel to a temporary farm to make the upgrade even faster. Monitor progress Upgrade logs is now split up so that there’s only one upgrade log per session, and a separate log for errors, making it easier to see how the upgrade went. The command line tools for upgrade now have status indicators that will visually show the progress of the upgrade. Also the upgrade status page in Central Administration (CA) tracks the progress and history of upgrades on the upgrade status page. Use the above to ensure upgrade process is on schedule, and be ready to “pull the plug” on the upgrade if you can see you are running out of time for the upgrade and need to recover the old installation. 2.1.5. Validate After upgrade is complete, you need to validate that the upgraded system really works. This means checking logs, checking rendering and checking that the database doesn’t have hidden issues. Upgrade event failures Reviewing the different logs associated with upgrade will give you a good indicator if everything really went fine. Look in the chapter Review log files above for more information. If issues are found, find out how to fix it, and restart or resume the upgrade! UI/UX issues Visually checking the upgraded farm will tell you if some of the functionality developed for the old version of SharePoint needs to be redesigned to look properly or even to work in SP2010. This includes HTML, CSS and JS issues, but could also be XHTML compliance issues. Also pages that fail to upgrade visually might be unghosted/customized in the old farm. You will then have to identify why the page was customized, determine if it is necessary to keep the customization, and then reghost the page in question. Data issues Check for orphaned items or database corruption using stsadm (see earlier chapter on orphaned items). Other data related issues are connectivity issues to data sources. Check that these work where used. 2.2. Visual upgrade By default the old look and feel of SP2007 and WSS3 is retained when doing an upgrade, but the site administrator has the ability to preview and change to the new SP2010 look and feel using the UI. When doing a db attach upgrade using stsadm.exe, setting preserveolduserexperience switch to true|false will enforce the UI accordingly. You could also automate the upgrade by utilizing PowerShell and/or the object model. For example using the SPSite.VisualUpgradeWebs method (consider including this code in a SPLongOperation since it, depending on the size of the site collection, could take a while to finish). In the ONET.XML of a custom site definition the UIVersion attribute in the Project element can be set to 3 or 4 to enforce UI version. 2.3. No International Domain Name support If you are upgrading a web content management site, and is using International Domain Names (IDN), it is worth mentioning that the support for IDN that was there in SP2007 was removed in SP2010! The only reason I have heard for this is, that “Support of internationalized domain names (IDNs) has been deprecated”. Not sure how to interpret that, but the fact is that it no longer works in SP2010, so if you used this in SP2007 you will need to delete all IDN settings in your SP2007 farm before upgrading. Note: See a full list of changes from SP2007 to SP2010 on TechNet http://technet.microsoft.com/enus/ library/ff607742(office.14).aspx 3. UPGRADING SOLUTIONS AND CODE 3.1. Recompilation Existing code that utilizes object model and runs within IIS will continue to work without recompilation (if compiled for AnyCPU or 64-bit). As when upgrading from SPS 2003 to SharePoint 2007 the upgrade process inserts assembly binding redirects from old assemblies to new assemblies (here 12.0.0.0 to 14.0.0.0) making the code automatically redirect to the new SharePoint dll’s. Code that runs outside IIS and utilizes the object model (workflows, feature receivers, timer jobs etc.) will either need recompilation or binding redirects to work with SP2010. Note: In SP2007 for a number of reasons it proved problematic to version assemblies. This often collided with both good development practice, and company rules on development lifecycles. These issues are well documented on the internet (including workarounds to get SPWebConfigModification class to add assembly redirects) so I won’t dig into that here. However the problem does no longer exist in SP2010 because that you now can specify assembly redirects directly in your solution manifest (see more below). It is not an automated process, so you will need to do it manually, but it is a lot easier than it was in SP2007, so developers should definitely consider using AssemblyVersion in code that is expected to have a long lifecycle! 3.2. Upgrading Custom Site Definitions As discussed in earlier chapter regarding visual upgrade, you can decide to keep the visuals as version 3 or you can decide to upgrade the visuals to version 4 adding the new layouts and tools such as the ribbon. This choice also affects how you want to upgrade your Custom Site Definitions (CSD). If you don’t plan on upgrading the visuals to version 4, most CSD should work as is, depending on how much is going on inside the CSD. In other words if you only used a CSD to add a new artifacts or change the basic layout of pages, you might be better off by using a new SP2010 site definition as a basis for re-creating that same functionality in the upgraded farm, or as close as you can get. Then add upgrade logic to your feature (see upgrading features later in this chapter). For more advanced scenarios, a better option would be to upgrade the functionality of the old site definition to match the new site definition. This involves changing the ONET.XML, since this has changed radically in the new version. 3.2.1. Upgrade definition files The purpose of Upgrade Definition Files (UDF) is to transform existing sites customized in the previous version of the product to take advantage of features in the new version. The UDF xml file maps custom lists, files and features from the old custom site definition to the new custom site definition during a schema or version upgrade. Though there are major changes to the product from SP2007 to SP2010, the paradigm shift isn’t as big as from SPS2003 to SP2007, where the feature concept was introduced, making the ONET.XML contain noticeably smaller. Hence the UDF for this version will be less complex, and most of the times not needed at all, depending on what customizations were done in the custom site template. The OOTB upgrade files for SP2010 can be found in 14\CONFIG\UPGRADE and can serve as a guide for upgrading your custom site definitions by selecting the site definition the custom site definition was based on. The custom UDF should be placed in the above mentioned folder and be given a unique name that begins with the name of the site definition (e.g. SPSNEWSCUSTOM_upgrade.xml). Note: For more information on upgrading Custom Site Definitions, check out “Architectural Approaches to Upgrading a Site Definition” http://msdn.microsoft.com/enus/ library/ms437476(v=office.14).aspx, “Upgrade Definition Files” http://msdn.microsoft.com/enus/ library/ms439232(office.14).aspx and “Upgrading a Custom Site Definition” http://msdn.microsoft.com/en-us/library/aa543837(v=office.14).aspx on MSDN. 3.3. Upgrading Solutions There are a few noteworthy changes in Solution packages regarding upgrades. In SP2007 it was tricky to add binding redirects in a consistent manner (SPWebConfigModification) since the runtime element is stored in another xml namespace (it could be done but it was tricky). Now this can be added declaratively as part of the solution manifest: <Solution …> <Assemblies> <Assembly DeploymentTarget=”GlobalAssemblyCache” Location=”MyWebPart.dll”> <BindingRedirects> <BindingRedirect OldVersion=”1.0.0.0” NewVersion=”1.1.0.0” /> </BindingRedirects> … </Assemblies> </Solution> This will add an assembly binding element to the web.config files for the assembly in question, redirecting code that uses the old assembly to point to the new assembly. Solutions can now also have dependencies declared in their manifest files. Three important things to note regarding solution dependencies though: solution dependencies does not automatically secure that dependent solutions are deployed. They just give you an error if you try to activate a solution that is dependent on another solution, and that solution isn’t deployed. Also you cannot have a farm based solution that is dependent on a user solution (Sandboxed solution). Last but not least: you will not receive any errors if you try and retract a solution that another solution is dependent on! 3.4. Versioned Features Upgrading artifacts within features were always a pain-point in SP2007. Upgrade scenarios for features in SP2007 would often mean adding a new dependent feature containing code in a feature call-out trying to change what needed to be changed in the feature. The good news is that upgrading features has received some attention in the new version, so it is now possible to upgrade features both declaratively and programmatically. You can even declare branches for different actions depending on version, or have element manifests being applied at update only. Up until now the version attribute in the Feature.xml manifest served no purpose. This has changed in the new version where the version attribute is used to detect if a given feature instance (SPFeature) needs to be upgraded. This is done by comparing the feature instance version with the feature definition (SPFeatureDefinition) version, hence securing that artifacts are identical whether it was just activated in a new version or upgraded from an old version. The upgrade behavior can both be defined declaratively in the feature.xml and in an event triggered when a feature is upgraded. 3.4.1. Declarative feature upgrade The declarative feature upgrade manifest contains an UpgradeActions element. Here you can declare actions that should only be applied for certain feature version ranges, including adding fields to content types, provision, move or rename files and more. Currently the following elements can be placed inside an UpgradeActions element: • VersionRange • CustomUpgradeAction • ApplyElementManifests • AddContentTypeField • MapFile The optional VersionRange element gives you the opportunity to target feature upgrade to specific version ranges (e.g. between version 1.0.0.0 and version 1.2.0.0). The declarative logic specified inside the VersionRange elements will then only be executed if the version falls inside the version range. This gives you the ability to branch upgrades with different behavior for each version. CustomUpgradeActions contains actions and parameters for custom code that is referenced in the UpgradeActions element (see more below under programmatic feature upgrade). CustomUpgradeActions can both be placed inside the UpgradeActions element and inside the VersionRange element. ApplyElementManifests is what you probably will use a lot when upgrading features: it will include an elements manifest that is only triggered on upgrade. This makes it easy to add new artifacts to an existing feature. The element can be placed under UpgradeActions or VersionRange elements. The optional AddContentTypeField makes it possible to easily add new fields to existing content types. By adding a PushDown=”TRUE” attribute to the element the change is pushed down from the site content types to every list content type. This was really a pain to do both declaratively and in code in SP2007, so that’s a really helpful change in SP2010. MapFile can be used to move or rename files during feature upgrade. 3.4.2. Programmatic feature upgrade The changes are not only declarative. There are several changes to the object model regarding feature upgrades. For once there’s now a FeatureUpgrading event that gets called for each matching VersionRange when a feature is upgraded. You can pass parameters to this event declaratively through the CustomUpgradeAction. New in SP2010 is also the Feature Upgrade Query Object Model. This can be used to query across farm to determine what features are installed and what versions they have, if they need to be upgraded, and then upgrade features accordingly. A QueryFeature method has been added to SPSite, SPContentDatabase, SPWebApplication, SPWebService and SPAdministrationWebApplication classes. These methods can be used to determine what features need upgrading in the relevant scope. Note: More on upgrading features and the feature object model on MSDN http://msdn.microsoft.com/en-us/library/ee535723(office.14).aspx To do the actual upgrade you call the Upgrade method on a deployed feature (SPFeature) and have it update to a new version. Note: The SPFeatureDefinition class already contained a version property. New in SP2010 is that SPFeature also contains a version property. This version does not necessarily correspond to the SPFeatureDefinition version: Upgrading a feature definition does not upgrade the feature instance itself. You will can use the query object model to obtain feature instances that need to be upgraded and programmatically call Upgrade() to upgrade to the new version. Read more on SPFeature version property here http://msdn.microsoft.com/enus/ library/microsoft.sharepoint.spfeature.version(office.14).aspx 3.5. Customizations against deprecated/changed UI Customizations done in Central Administration and SSP Admin UI will also have to be reimplemented. Central Administration has been completely restructured, and SSP has been replaced completely, so configuration links won’t show up as expected. Since the HTML and CSS has changed in the new versions, depending on the layout the customized pages will look different in the new UI, even if the UI was done carefully emulating the existing configuration pages using the same controls! If these links are still needed, they should be moved prior to an upgrade. For application pages consider changing the MasterPageFile attribute with the DynamicMasterPageFile attribute. This will make the application page reference the site master page rather than application.master. 3.6. Security changes 3.6.1. Web Parts As with SP2007 ASP.NET web parts should be preferred. WSS web parts while being phased out are still supported, but there are really no good reasons to use them anymore: Web Part Page Services Components (WPSC) that was part of WSS web parts would allow you to do client-side connections, but the new feature in SP2010 called Client Object Model exceeds anything WPSC would ever allow you to do. Also AJAX (including postbacks) is now natively supported. Other reasons to use SharePoint web part classes include part cache, but this can easily be solved in ASP.NET web parts using runtime cache. Note: More on Managed Client Object Model on MSDN http://msdn.microsoft.com/enus/ library/ee537247(office.14).aspx The Client Object Model is also the reason that new Cross-site scripting (XSS) safeguards have been implemented in SP2010. Properties in web parts that can be changed by contributors, combined with Client Object Model are a XSS risk. This is why custom properties in web parts now require at least Designer level (previously it only took Contributor level). The new XSS safeguards are the RequiresDesignerPermissionAttribute that can be applied to properties in web parts and SafeAgainstScript safe control. Both are designed to limit access to viewing and saving properties in web parts. Note that all web parts are affected by these new security measures (including old SP2007 web parts). This means you should review existing web parts to check if this new restriction breaks functionality, validate the risk of XSS and evaluate if you can risk setting the SafeAgainstScript SafeControl to true (false is default!). Note: XSS Safeguard only affects shared web parts, not Personal or personalized properties. 3.6.2. Sandboxed Solutions Sandboxed solutions are a new concept in SP2010. Sandboxed solutions address a common problem in SP2007: you would have farm administrators would like to keep their servers up and running with good response times, and secure from malicious code. But you would also have developers that were told to develop custom functionality. Testing code before deploying it to a farm is both time consuming and difficult. Even with several test levels such as unit tests, smoke tests, functional test, load test and integration test, you will often not discover problems with the code until it is too late: in your production environment. Sandboxed solutions is a subset of a standard solution: it is limited both in regards of object model and performance to run within a process called User Code Service (SPUCWorkerProcess.exe) that runs within a very limited Code Access Security policy (wss_usercode.config, that should not be edited!) and only on selected servers in the farm. It also uses a limited subset of the SharePoint API (reflected in Visual Studio intellisense). A solution that runs within the sandbox is monitored on an array of metrics such as CPU, queries to database, unhandled exceptions etc. You can set up quota limit that the code needs to stay within. If this quota is exceeded, warnings will go out to Operations and when a limit has been reached, the code is temporarily disabled. You can build custom solution validators that allows only certain types of artifacts (e.g. web parts) or code signed with specific signatures While th is new concept makes a lot of sense, it also means that you need to be aware of this when you upgrade your existing solutions: Code-wize you will need to review your solutions, so that they will still work within the solution sandbox, since sandboxed solutions run against a subset of the API and with a limited CAS policy, a lot of the stuff you did yesterday (like web service calls, or calling code that is not marked with AllowPartiallyTrustedCallers=True) will no longer work! It is possible to make calls to the “real” API, but it requires you to move the code to what’s called a full trust proxy in a separate assembly that goes in the GAC, and call the proxy from the sandbox. You can choose to ignore sandboxed solutions and just upgrade your old 2007 solutions as what is now called Farm Solutions, but all in all the concept of sandboxed solutions will need to be addressed before upgrading a farm. There are good reasons to use the sandbox, including improved security, better monitoring and in the end a more stable and better performing farm, and looking ahead, all new development that fall inside what can be achieved as sandboxed solutions should be developed as such! With regard to architecture, it should be considered to dedicate server(s) to run sandboxed solutions further isolating custom code from the rest of the farm. Note: For more information on SandBoxed solutions check out: http://msdn.microsoft.com/en-us/magazine/ee335711.aspx http://blah.winsmarts.com/2009-12– SharePoint_2010_Sandboxed_Solutions__The_Definitive_Guide.aspx For more information on custom solution validators check out the API: http://msdn.microsoft.com/enus/ library/microsoft.sharepoint.usercode.spsolutionvalidator(office.14).aspx For more information on full trust proxies see the API: http://msdn.microsoft.com/enus/ library/microsoft.sharepoint.usercode.spproxyoperation(office.14).aspx 3.7. Large List Throttling There’s a new performance related feature in SP2010 called Large List Query Throttling: Queries that touch large lists will fail based on predefined thresholds set in CA. There is a good chance that this could cause problems for legacy code, especially if development is being done as an administrative user! Also if development and test environment does not have realistic data volumes, code could fail without this being caught before deployment. For this reason you should start developing against least privileges, and always try to have as realistic data as possible in your environment (for lists it would even make sense to have lists that are a lot larger than in production). Even if you only select a small subset of items from a large list, the API and database still need to do a table scan to select the appropriate items. Hence a small query on a large list will be throttled and throw an exception. This can be resolved by adding an index on the list that matches the field that is used in CAML query to filter the list. It is possible to override the Resource Throttling: SPQueryThrottleOption.Override if Object Model Override is set to Yes in CA and if the user executing the query has Full Read permissions. Note: To avoid Yellow Screen of Death (YSOD) code needs to be changed to log and catch a new exception SPQueryThrottledException. 3.8. Deprecated API’s When you recompile your old SP2007 code for SP2010, you will see warnings for types and methods that have been deprecated in SP2010. Most of these will continue to work without breaking anything in SP2010, but you are encouraged over time to upgrade the code since Microsoft no longer will invest in these API’s. Note: Get a list of deprecated types and methods made obsolete in SP2010 and SP2007 on MSDN: http://code.msdn.microsoft.com/sps2010deprecated Chris Auld mentioned a plug-in for Reflector that would catch obsolete methods and warn against code that could have problems in SP2010, for example in relation to sandboxed solutions. Tool should become available at http://www.syringe.net.nz/blog 3.9. Hardcoding issues If you have hardcoded references to anything residing in the old 12-hive (aka SharePoint root folder: c:\program files\common files\microsoft shared\web server extensions\12) these should be updated to point to the 14 folder instead! 3.10. Upgrading the look & feel to the new version If you choose to go with the new visual upgrade like the Ribbon, developer dashboard etc., you need to manually add these controls to your master pages and page layouts. After upgrading the solution to SP2010, in Site Settings > Site Collection Administration select Visual Upgrade > “Apply the new User Interface to All Sites”. Click Update All Sites. This will change the appearance to the new interface. While the site settings page itself properly will upgrade without issues, but if you are using a custom site definition, you will need to manually replace the old UI controls: Since SharePoint distinguish between v3 (SP2007) and v4 (SP2010) master pages (v3 master pages are filtered out in the standard Master page view), start by creating a new blank v4 master page using SharePoint Designer 2010 (SPD) and replace the content with the content of the v3 master page. • Delete the page editing toolbar (PublishingConsole) tag prefix and associated controls • Delete site action (PublishingSiteAction) tag prefix and associated controls (including the SPSecurityTrimmedControl wrapper control) • Add core.js if not already present as a ScriptLink control • Copy the ribbon DIV html and control (SPRibbon) from v4.master and paste it into the new master at the very top of the body (inside FORM element) • Add register tag prefixes for ribbon (MUISelector) • If you use breadcrumb control, this is contained in ribbon, so remove control and surrounding HTML from master • Copy the developer dashboard control (DeveloperDashboard) from v4.master and insert it into the bottom of the body of the new master Note: Further customizations can be done (such as maintaining the position of the ribbon while scrolling) info on upgrading an existing master page to the SharePoint Foundation master page can be found on MSDN: http://msdn.microsoft.com/en-us/library/ee539981(office.14).aspx 3.11. Upgrading projects to Visual Studio 2010 Part of upgrading your code should be migrating from VS2005/VS2008 to VS2010. There are a bunch of new cool features for SharePoint in the new VS2010, so it is recommendable to upgrade existing projects to the new development platform. Also it will make upgrading existing code easier. If your projects were created using VSeWSS you can download a VS2010 template that will upgrade your projects to VS2010 SharePoint projects. After migration you will need to manually consolidate your artifacts using Feature Designer and Packaging Explorer. Note: The VSeWSS upgrade tool is not officially supported by Microsoft. You can download Visual Studio 2010 (Beta) migration tool for VSeWSS SharePoint projects here: http://www.microsoft.com/downloads/details.aspx?FamilyID=41019A15-8C73-497C-97FB– 502A619A6C46&displaylang=en If you use other tools like STSDEV or WSPbuilder, you can consider a number of different more or less manual approaches: The first approach is a manual approach where you basically build your project structure up manually importing code and artifacts as you go: • First you need to evaluate what your visual studio projects contain. o If you have separated your different logic into tiers for data access, business logic and presentation, there is a good chance that these class library projects can be copied directly into VS2010. o For visual studio projects containing artifacts create an empty SP2010 project. Here you must choose between creating a sandboxed solution or a farm solution –the choice will depend on what customizations are done in the project, since sandboxed solutions put a lot of restrictions on what can be done. Sandboxed solutions should be preferred, but will probably require a lot more effort on refactoring the code to keep within the sandbox boundaries. • Use the new VS2010 feature called Mapped Folders to map the SharePoint root (aka 14- hive) folders you need for your project. Add your existing artifacts into the relevant folders o To take full advantage of VS2010 you can also create some of the artifacts (such as web parts) from scratch using the corresponding template and then copy/paste the code and declarative xml from your existing files. • VS2010 now has a feature called Replaceable Parameters that basically are tokens that are replaced after manifest transformation. The tokens are extendable and include tokens for things like $Sharepoint.Project.AssemblyFullName$ Consider replacing • For artifacts that need to be provisioned to document libraries you create Modules and add your existing content to the modules. • Features can either be created manually or added through the feature Manifest Template (<featurename>.Template.xml). The features added through designer and Manifest Template is merged into a single manifest file for the feature. • Add the artifacts to the Package (Package.package file in project folder) using Package Explorer or Package Designer. • For artifacts currently not supported by VS2010 (for example custom site definitions) add the relevant xml from your existing manifest.xml files to Package.Template.xml that can be found nested under Package folder. Artifacts listed in Package.Template.xml are merged with Package artifacts during packaging into a single solution manifest file. Note: Read more on MSDN about Packaging and Deploying SharePoint Solutions: http://msdn.microsoft.com/en-us/library/ee231544(VS.100).aspx Read more on MSDN about the structure and files in SharePoint project types: http://msdn.microsoft.com/en-us/library/ee476619(VS.100).aspx#projectcomponents To ease this manual process you can instead choose to import SharePoint solution packages (WSP) into VS2010 using the Import SharePoint Solution Package project type. As of now this template works best for simple WSP packages, but hopefully it will become better in the final release: • First create a WSP file containing the artifacts you need to migrate to VS2010. • Create a new Import SharePoint Solution Package project in VS2010 and select WSP file when asked. • If not supported artifacts was contained in the WSP you might get a warning, but don’t count on it. The import still has a lot of beta hiccups, so for example custom site definitions disappear after an import and so does assemblies for CAG. I don’t know if this will be fixed for the final release, but still the tool is still a huge help when you want to convert existing projects. Note: Carsten Keutmann the author of WSPBuilder also has released a beta of WSPBuilder for VS2010. I haven’t had time so far to check this out, but it is available on Codeplex here: http://wspbuilder.codeplex.com/releases/view/30858 The third way of importing a project would be to “roll your own” import tool. VS2010 has specific interfaces defined for creating extensions of various kinds. For example the ISharePointProjectFeature interface for adding items to features and the ISharePointProjectPackage to add items to packages. Note: Since the SharePoint Tools in VS2010 are extendable, we already see a lot of tools by the SharePoint Community. So far most notably Community Kit for SharePoint: Development Tools Edition that contains several enhancements focused on deployment, artifacts and more. CKS:DEV can be found on Codeplex: http://cksdev.codeplex.com/ There are a lot of good reasons to upgrade to VS2010 like F5 debugging, templates for specific tasks, native support for solutions and features, possibility to browse SharePoint sites using Server Explorer. The list goes on! All this makes SharePoint development a much better experience than developing in earlier versions of VS. Note: More info on importing WSP into VS2010 on Channel9: http://channel9.msdn.com/posts/funkyonex/Importing-SharePoint-Solution-Packages-WSP-into– Visual-Studio-2010/ For more info on what’s new in VS2010 with regards to SharePoint development read this TechNet article: http://msdn.microsoft.com/en-us/library/ee290856(VS.100).aspx 3.12. Client upgrades Be aware that Internet Explorer (IE) 6 no longer is supported for authoring, due to its poor interpretation of web standards. As part of an upgrade you should plan for upgrading to a supported browser. Note: Read more on TechNet: Plan browser support http://technet.microsoft.com/enus/ library/cc263526(office.14).aspx 4. PLANNING Now that the basics for upgrading SharePoint 2007 to SP2010 have been laid out both regarding servers and code, it is time to think about what the specific actions should be when doing an upgrade. This chapter only contains general recommendations, as the approach will be dictated by external factors such as if the company that pays for the upgrade is willing to buy new hardware for either a full db attach upgrade or a hybrid approach involving new hardware. Also the physical design of the solution, the size of the content databases, and the amount of customization on the farm will affect the recommended approach, along with demands for downtime. 4.1. Planning prerequisites First thing that should be done is bringing the solution in a supported upgradable position. This includes upgrading any OS used as SharePoint servers from 2003 to 2008 server R2 for all involved servers. Make sure SQL server is running 64-bit with latest SP and CU. For SQL Server this is SP3 with CU3. Optionally consider upgrading to SQL server 2008 SP1 with CU 2 since SQL Server 2005 support lifecycle is terminated in 2011 (http://support.microsoft.com/lifecycle/?p1=2855) another reason for upgrading is improvements from 2005 to 2008 including better compression, better encryption, higher availability through improved patching capabilities, throttling and improved locking mitigating blocking issues, better mirroring, support for Remote BLOB Storage etc. (read here http://blogs.msdn.com/mikewat/archive/2008/08/19/improving-sharepoint-with-sql-server– 2008.aspx and here http://www.sharepointjoel.com/Lists/Posts/Post.aspx?ID=297 for more). Also consider upgrading the OS that SQL server is running on to Windows 2008 Server, since no further service packs is considered for 2003 server (http://www.microsoft.com/windows/lifecycle/servicepacks.mspx). When looking at upgrading the software, also consider upgrading or replacing existing hardware: The upgrade process itself will demand more hard disk space to instantiate a number of new databases, existing databases will grow and log files will take up space as well (transaction model is automatically set to Simple for databases though). 4.2. Planning upgrade model When choosing the appropriate upgrade model, several things will affect your choice: For example consider if the servers are already within specifications or if you can expect acceptable performance by upgrading the hardware (scale up)? If this is the case, this speaks for doing an in-place upgrade. On the contrary, if we already now can see that the existing hardware must be replaced, this will necessitate a db attach upgrade. Another question to be asked are if you have scripted installs. If you don’t have this, this could speak for an in-place upgrade, rather than having to do a manual install, that is prone for human error. Are customizations as a general rule structured and reproducible (read: solutions and features)? If not, this speaks for doing an in-place upgrade to avoid the process of reproducing customizations on a new server. Also ask yourself what is acceptable downtime? If downtime is totally unacceptable, favor solutions that mitigate downtime, such as read-only databases. As described in the chapter on upgrade models, there is also the possibility to choose a hybrid model. For example the read-only databases hybrid approach has a lot speaking for it, with regards to downtime mitigation. In general in-place upgrade is considered risky, since you won’t be able to easily recover from a failed upgrade. If getting new hardware is out of the question for the upgrade, be sure you have a tested disaster recovery plan that will enable you to re-build your SP2007 farm if need be. 4.3. Planning new Server Architecture Since the architecture on SP2010 has changed quite a lot compared to that of SP2007, you also need to take this into consideration when doing an upgrade. Decide how the service architecture should be: Should new server roles be added to the farm by adding new hardware or by combining roles on existing servers? Would the farm architecture benefit from isolating certain services, since this is now possible in SP2010? Default in SP2010 is that all services are disabled. This is good since it indicates that you should consider for each service if it should be enabled. Sandboxed solutions Consider isolating Sandboxed solutions on a separate server (remote mode). Remote mode is more scalable, but requires more administrative involvement. Note: Further information on planning sandboxed solutions, including planning resource usage quotas on TechNet http://technet.microsoft.com/en-us/library/ee721991(office.14).aspx Remote Binary Large Object Storage (RBS) SP2007 used integrated storage architecture for Binary Large Objects (BLOB), meaning that the BLOB was stored in the content database along with the metadata. As content databases grow, so does the time it takes to backup and restore data, hence affecting the Service Level Agreement (SLA) of the farm. In SP2010, it is possible to store BLOB data separate from the content database using RBS. This allows for storing BLOBS on cheaper storage and has faster backup/restore from SQL server since metadata is stored separately from BLOB’s. RBS defines an interface that allows external BLOB storage providers to support it. In the time of writing there are 5 external providers that either already integrate to, or is in the process of writing providers to integrate to RBS: EMC², OpenText, NetApp, AvePoint and CommVault. Note: RBS should not be considered a silver bullet for keeping disaster recovery within SLA, but rather as a specific tool for a specific problem. Also consider that the whole backup/restore picture will be complicated by having to fetch data from several locations. The SQL Filestream RBS provider that SP2010 provides out of the box is supported by both SharePoint and SQL backup and recovery, but support for backup is up to the individual RBS provider. RBS has several advantages over the existing alternative in SP2007, External BLOB Storage (EBS): • It has a managed interface with a provider API • The scope for setting up RBS is per content database, so you can configure one BLOB store provider for one content database and another BLOB store provider for another (in EBS you had farm scope). • As a consequence of the above, you can have many providers with RBS, where EBS only supported one provider • You can configure a RBS maintainer to support retention policies, detect orphans etc. • RBS can be configured through the UI and using PowerShell • You can migrate BLOBS from one store to another using PowerShell Note: Using RBS requires that SP2010 runs on SQL Server 2008 R2. The existing architecture in SP2007, called External BLOB Storage (EBS), is still supported in SP2010, but should be considered deprecated 4.4. Test, test, test As described earlier, upgrading is very much a trial and error discipline. You cannot expect to upgrade a complex farm with lots of content, customizations and configurations perfect the first time. Even if this is possible, you have no way to tell how long the process would take. Practicing the upgrade process documenting the farm and customizations along the way, will give you a much better gut feeling when you do the actual upgrade: You will have a good idea on what to do, since you already have done it plenty of times, you will have a certain degree of knowledge about the outcome of the upgrade, and even if something should go wrong, you have documentation ready to recover your old farm if need be. Using virtual environments to replicate farm setup, where you test for issues after upgrade. If possible consider doing a pilot, where only part of the farm is upgraded and let end users test the site extensively for you with everyday usage. 4.5. Planning operations scheduling Plan upgrade over a weekend. This will give you time to roll back if something breaks in the upgrade process. A simple schedule can help you determine if you are on track or if you should consider rolling back the original site: • Friday 18:00 start backups • Saturday 0:00 start upgrade of content farm/databases • Sunday 12:00 upgrade must be effectively complete, or rollback must begin • Monday 06:00 environment must be up and running Scheduling should also include a plan for operations staff that should be available during the actual upgrade. 4.6. Planning code upgrade approach In parallel with the planning and trial upgrade of the farm, the development team should be looking at what to do with the existing customizations. This could be done as a separate test upgrade, where solutions and features are installed on a test SP2010 environment and tested. Some things to consider regarding existing solutions, features and code: • Should code be migrated as farm solutions, or should an effort be made to convert the solutions to sandboxed solutions? • Should obsolete namespaces, types and methods be addressed? • When upgrading features consider using the new possibilities available (e.g. new fields in Content Types). • Does code access large lists, or could lists grow outside specified throttling metrics? Treat code accordingly, and decide how to handle throttle exceptions. • When reviewing solutions, features and code, think about if the functionality is still relevant -it could either have been replaced by OOTB functionality or the functionality it was addressing could have been removed from the platform (e.g. custom links in SSP). • Code that run outside IIS should be recompiled with new SharePoint assemblies or binding redirects should be defined along with AssemblyVersion. • Considering the wealth of new features in VS2010 for developing and deploying SharePoint code, migrating your projects to VS2010 should have a high priority Note: Download content posters for SP2010 (including 4 posters on upgrade) here: http://blogs.technet.com/tothesharepoint/archive/2009/10/23/3288841.aspx 4.7. Planning user adoption Finally you should plan for your end users. SP2010 is an awesome product, but it is also huge and a lot of the ways things was done in SP2007 has changed in SP2010, especially when enabling visual upgrade. Examples include the Ribbon, new templates for Information Workers, and a new and vastly improved SharePoint Designer to mention a few. Training your site administrators, designers and contributors will prove valuable before doing the actual upgrade ensuring end user adoption from the start. Note: There are a lot of online resources for end user training, a lot of who are free. The below link is an example of free online videos to train end users in SP2010: http://www.point8020.com/SharePointEndUserTraining.aspx