[Important News!]
I’ve kicked off a new project/band where I’m creating music with a heavier techno/metal sound called binarywaste. You should check it out by clicking right here!
Windows SharePoint Services 3.0 (WSSv3) and Microsoft Office SharePoint Server 2007 (MOSS2007) now support the ability to create folders within lists. This is quite frankly one of the coolest features in this release of the products (well, aside from all the OTHER coolest featuresJ), but the point is, it also adds a layer of complexity to how you enumerate lists now if you want to traverse a List folder hierarchy the same way you might want to enumerate a Document Library folder hierarchy.
This is something that was giving me a real headache trying to figure out how to do all day long. (Strangely, the solution I came up with I thought about a couple of days ago, I just didn’t want to have to do it this way, nor did I think I would “Have” to do it this way). I searched around, referenced the SDK, searched around more, but still came up blank. After collaborating with a lot of my peers and friends (And for those of you I pinged on this, thanks for your suggestions), I found that the suggestions they were sending me, were paths I already took and found they would not work J But, just after I harassed them, I realized what the solution was J
What I’m going to do first, is provide you a bit of background on what I attempted, and show you why those solutions don’t workJ. Then show you the solution that will work against both Lists and Document Libraries, or any list in generalJ.
Traversing a Document Library (In WSSv2 or WSSv3) was a simple process. Consider the following code if you were enumerating the document libraries using the lists collection of a SPWeb
// Within some method somewhere
foreach(SPList list in web.Lists)
{
if(list.BaseType == SPBaseType.DocumentLibrary)
{
//Traverse the folder
TraverseFolder(list.RootFolder);
}
}
TraverseFolder might be implemented something like this:
public
long TraverseFolder(SPFolder folder)
{
// Do something interesting on the folder
// Perhaps sum up the file sizes
long lFolderSize = 0;
try
{
foreach (SPFile file in folder.Files)
lFolderSize += file.Length;
if (bRecurseFolders)
{
foreach (SPFolder subfolder in folder.SubFolders)
lFolderSize += TraverseFolder(subfolder);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
return lFolderSize;
}
It would be nice if you could do the same thing with a base SPList, but you can’t J
You may be inclined to start with something like this, the same way I was inclined to do so J
foreach (SPListItem item in list.Items)
{
Console.WriteLine(“List item: “ + item.Name);
Console.WriteLine(“List item type: “ + item.ContentType.Name);
}
// Folders are not counted in list.Items, so we have to look at those seperately.
// But, the Folders collection is just another collection of SPListItems.
foreach (SPListItem item in list.Folders)
{
Console.WriteLine(“List Folder item: “ + item.Name);
Console.WriteLine(“List item type: “ + item.ContentType.Name);
TraverseListItem(item);
}
But there are a couple of problems with this approach
- The items returned from via SPListItemCollection at list.Items, returns ALL items over the whole list hierarchy
- The items returned from via the SPListItemCollection at list.Folders, returns ALL folders over the whole list hierarchy
So consider a list that was structured this way:
- Announcements
- Item 1 in the root of the Announcements list
- Item 2 in the root of the Announcements list
- Folder 1 in the root of the Announcements List
- Item 1 in Folder 1
- Item 2 in Folder 1
- Folder 2 under Folder 1
- Item 1 in Folder 2
- Item 2 in Folder 2
The enumeration of list.Items above would return the following items:
- Item 1 in the root of the Announcements list
- Item 2 in the root of the Announcements list
- Item 1 in Folder 1
- Item 2 in Folder 1
- Item 1 in Folder 2
- Item 2 in Folder 2
And the enumeration of list.Folders above would return the following items:
- Folder 1 in the root of the Announcements List
- Folder 2 under Folder 1
Let’s assume you have not discovered that yet and when you detect that the item is a folder, you try to treat it this way J
You’re TraverseListItem () call may be implemented something like this:
private
static
void TraverseListItem(SPListItem listitem)
{
SPFolder folder = listitem.Folder;
if (folder != null)
{
// This item is a folder, therefore call on contained items recursively.
Console.WriteLine(“Found a folder!!!”);
Console.WriteLine(“Folder: “ + folder.Name);
foreach (SPFile file in folder.Files)
{
// Do something
}
}
}
The problem is that even though the list item is a folder, and you get a valid SPFolder
back from listitem.Folder, but unlike a Document Library, it will have a count of zero.
But hey! There is a listitem.ListItems collection! That would work right?
private
static
void TraverseListItem(SPListItem listitem)
{
SPFolder folder = listitem.Folder;
if (folder != null)
{
// This item is a folder, therefore call on contained items recursively.
Console.WriteLine(“Found a folder!!!”);
Console.WriteLine(“Folder: “ + folder.Name);
foreach (SPListItem item in listitem.ListItems)
{
// Do something
}
}
}
No, that collection returns the base SPListItemCollection from the list itself, just like SPList.Items and this would get you into a never ending loop, as you’d just recurse the same items, over and over, and over, and over,
Ok, so maybe you rethink it all, and decide to treat the list as a SPDocumentLibrary
and do what you know works. But no, the base type of a List is GenericList, not SPDocumentLibrary, therefore something like this won’t work:
SPDocumentLibrary docLib = (SPDocumentLibrary)web.Lists[“Announcements”];
So, just how the heck to you enumerate the folder items in a list when you discover that the item you’re looking at is in fact a folder?
It’s actually very easy, but I would vote it’s a horrible way to have to go about it.
Enter SPQuery:
private
static
void TraverseListFolder(SPFolder folder)
{
// Get the collection of items from this folder
SPQuery qry = new SPQuery();
qry.Folder = folder;
SPWeb web = null;
try
{
web = folder.ParentWeb;
SPListItemCollection ic = web.Lists[folder.ParentListId].GetItems(qry);
foreach (SPListItem subitem in ic)
{
Console.WriteLine(“List item: “ + subitem.Name);
Console.WriteLine(“List item type: “ + subitem.ContentType.Name);
if (subitem.Folder != null)
TraverseListFolder(subitem.Folder);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
// Cleanup that nasty web reference.
if (web != null)
web.Dispose();
}
}
Ok so cool, we now have a way to get ONLY the items in a folder within a list at any level without introducing any items that are not in scope.
So, we’ll just use this method, and to traverse the list, but hey, where do you start? Well, fortunately SPList exposes a RootFolder property that gives you a SPFolder object to start with J
So we can write our List traversal Kick off method as such:
private
static
void TraverseList(SPList list)
{
Console.WriteLine(“Traversing list: “ + list.Title);
Console.WriteLine(“Base Type: “ + list.BaseType.ToString());
TraverseListFolder(list.RootFolder);
}
Simply pass in a reference to a list, and away we go. Very compact, and it works regardless if the list being passed in is a Document Library or any kind of list.
Hope this helps!
- Keith Richie
Keith Richie 
I wonder what the cost of querying/requerying the list for each folder/subfolder will cost just to iterate what should have been simple to hierarchy to traverse. Yuck!
We need to introduce Joe Celko’s concepts like nested sets for hierarchies (http://www.developersdex.com/gurus/articles/112.asp) to the Microsoft product teams.
🙂
I agree… I too am wondering what the cost is going to be. I won’t argue that it’s probably not the most performant.
Considering that a SPFolder.Files reference is going to make a trip to the database, and the call to web.Lists[folder.ParentListId].GetItems(qry); is goint to make a trip to the database, It boils down to running a SQL profile trace to see how chatty one way or the other is.
That would give us a comparison for “Document Libraries”, but not for lists :).
Thanks for the detailed explanation. Another vexing question is – how do you move list items from one folder to another. There is no way (that I can find anyway) to do it in web UI. There is no Explorer view option available for the Lists like you have with Doc Libs. On publishing sites, you can go into the Site Manager page and select list items and choose to Move them, but this is not available to the average user.
As for via the OM, the only way I’ve come up with so far is to create a copy of the list item at the new location, and then delete the original list item. Of course when you do this you lose the original item’s Creator, Modifier, Created, Modified values, which IMHO is wrong.
SPListItem doesn’t seem to have the expected/needed MoveTo() method.
The Folder and Url properties are read-only. 😦
I’ll appreciate to know if anyone figured out a better approach.
Now that’s a good one :), and not one I have an immediate answer for 🙂
So SPListItem.Copyxxx() doesn’t preserve the Creator,Modifier, etc. Well I can see why it wouldn’t do that, as it’s a “Copy” and I’m sure that the copied item now shows the user context information that your code is running under right?
What about getting a folder reference to the folder containing the item and call SPFolder.MoveTo() ? Would that help?
Of course, that means that it’s all or nothing for the folder, and I would still see a need for individual item moves.
1.) What are the command line args for the following Sharepoint code
“Sample code to either Calculate size of all files or Export Files from a web”
I think its url -export some_third_arg???
2.) How do I install the program after I have built the exe file?
This is the URL that I am referring too.
http://blogs.msdn.com/krichie/archive/2005/02/15/373271.aspx
Thnaks,
Dale
Hi,
SPList.Folders property will not return a colection containing all the folders and subfolders in that list?
Yes it will, that’s exactly what it does, and what I refer to in this post when I state:
“…the enumeration of list.Folders above would return the following items:
Folder 1 in the root of the Announcements List
Folder 2 under Folder 1
”
I gives you back a flat set of the folders within the list or document library, but does not give you heirarchy nor show containment.
I was using the SPList.Folders property but I got some issues with lists that were imported from list templates and have custom folder content types.
The folders are not apearing into the Folders collection until I edit them manualy in SharePoint.
Any ideea why?
It could be (And this is just a guess) that when you imported them, you imported them but they were not checked out, and the user account being used to enumerate SPList.Folders is only seeing the checked in versions.
I suspect that when you edit them manually, they are being checked in, and then discoverable?
hi,
Is there any webservice method to get items from list recursively from folders and subfolders at oneshot.
thanks,
surya
Hi I am having an issue with the explorer view: hoping you can explain it.
Any document loaded had a created and modified date, in the All Items View – if you modify the doc it changed the modified date, but in the explorer view when you right click and say properties, the created on and modified on date stays the same. Can you explain why to me please?
Has anyone come up with a way to copy or move list folders along with their content? I have tried and failed so far. When ever I call the MoveTo method on a SPFolder I get the exception: “Folder //destination/source folder name// does not exist”
I would think that the source folder would be created there as the destination folder without instruction.
Can anyone help me out here?
Has anyone figured out how to traverse the list BACKWARDS? For example, if I have a reference to a particular folder, how can I traverse the list back up to the root folder? I’ve searched everywhere and not found any mention of this. Any help would be greatly appreciated!
Have you tried walking the SPFolder.ParentFolder property until it is null?
If the point is just to query items in a library recursively (without folders), why not just use GetItems and set the Scope attribute to recursive in your SPQuery.ViewAttributes property?
myQuery.ViewAttributes = “Scope=\”Recursive\””;
This seems tremendously more efficient than looping.
That would probably work, but I needed to ensure that I only retrieved the immediate children of a given level, rather than the entire batch of items .
I’m sure I could do this, then do some logic to parse it out after the fact, but this solved my problem at the time. Thanks for the great suggestion though!
Thanks for this article. Obviously you put a lot of work into it.
I’ve got a couple of things I’ve used:
SPFolder.SubFolders: for a list this returns the correct subfolders of a given folder, starting with SPList.RooFolder, even in a custom list.
Moving SPListItem objects with SPFile.MoveTo – use this to move SPListItems.. It seems jerky, because logically a simple SPList – SPListItem object is not a file.. but actually it works. Nevertheless there is a problem – I think SPListItem objects return null for .File. So you can get around this with:
SPFile item_file = dm.thisWeb.GetFile(dm.thisItem.UniqueId);
item_file.MoveTo(NEWURL);
… but you have to do a GetFile() for each listitem. The whole thing just shows how stupid it is to implement a folder structure for objects and then leave out simple folder-related tasks like moving, listing, etc. (or the nonexistant SPListItem.ParentFolder object..) direct simple folder queries to the Database would be much more optimal.
Basically with a simple SPList you use SPList.RootFolder and SPFolder.SubFolders to traverse the list, but then only display a single folder’s content using the query.
Another way to get all list items within a certain SPFolder hierarchy is to query with MyFolderServerRelativeUrl</BeginsWith or some such query (there are quite a few ListItem properties cotnaining the URL) or to compare the SPListItem.Url property with your folder url –
SPListItem.Url.Substring(0, SPListItem.Url.LastIndexOf(“/”)) == myFolder.ServerRelativeUrl.. or similar.
Why do we have to FIND OUT and GUESS how SharePoint works.. pity they didn’t just make it work intuitively and document it properly from the start.
Thanks a lot. “Moving SPListItem” solution is wonderful. Thanks again.
Pinging back.
Thanks for this.
Excellent write up. Will be very helpful in what I am working on.
I have a semi-related question. Being new to SP, I noticed that when I use foreach to enumerate over the Items in a List, that I get huge memory bloat that does not release. Is there a way around this? What is causing it?
Any help is appreciated.
Not sure the direct calls your making during the enumeration, but if you are accessing the lists ParentWeb, it will instantiate a new SPWeb instance that you should call dispose on.
See: “Best Practices: Using Disposable Windows SharePoint Services Objects”
http://msdn2.microsoft.com/en-us/library/aa973248.aspx
Article was written for WSS 2.0/SPS2003 but still applies to WSS 3.0/MOSS
Keith, thanks for the quick reply. I will take a look at the article, but I am already properly disposing of my Web and Site references.
Is there a way we can take this conversation “off-line” so as to not clutter up your blog?
Keith, is it possible to add a property to a list folder? I’m asking this because I need to show the items and folders of a Sharepoint list in an application outside Sharepoint. They have to be shown in a specific order, something like this:
Item #1
Folder #1
Subitem #1
Subitem #2
Subitem #3
Item #2
Item #3
Folder #2
Subitem #1
Subitem #2
Item #4
Item #5
So I thought of solving this by using a property (or something of the sort) in the folder, just like a column in the list item, to store the display sequence number.
How can I do that?
I believe you can. You just need to create a new content type based on the folder content type, and add the columns that you require to the folder.
DanMc:
>>Being new to SP, I noticed that when I use foreach to enumerate over the Items in a List, that I get huge memory bloat that does not release.
If you use the SPList.Items collection it will theoretically cache ALL the items in the list in memory (SPList objects). In lists with thousands of items this is not optimal, and in large lists it could lead to memory clutter. Much better is to use Views and Queries to return only the required items. I think you can divide the query into batches by setting the SPQuery.ListItemCollectionPosition and SPQuery.RowLimit properties. Basically it all boild down to using the Items collection as little as possible. us ItemCount to get the number of items.
Claudia:
>>is it possible to add a property to a list folder?
Each SPFolder object in a custom list has a corresponding Item object accessible through SPFolder.Item or SPList.GetItemByUniqueId(SPFolder.Item). You can get the folder’s properties using this SPListItem object just as you would with any other list item.
I have folders around 3000 in my shared document library & it seems very slow when we use via webservice…
how can we tune up this folders in sharepoint to be fast response.
performance is needed right now!
i had created webservice to get site defention which retrives all folders & path enumerating.
is that 3000 folders at the same level?
Personally, using folders for document navigation limits flexibility. I prefer using tags, then creating views for the appearance of folders. Then you can have the same items appear in multiple “folders (views)”
I only use folders for security. Then use “show all items without folders” for all my views. If we have line items or documents that only a certain group should see they are added to the folders that set the security. Then everyone can look at the same view, but only see the items they have access to.
Using folders in lists or libraries is unfortunately a bad habit but it occurs… therefore thank you very much for your explanation.
Do you know if this methods are still valid for SharePoint 2010 or is there an easier way to iterate through folders in lists and libraries?
Andreas
Andreas, afaik…this is still valid for 2010.
@Andreas : A lot of people don’t like folders… But sometimes you can’t avoid them : when a company use SharePoint to store a really large amount of items (sometimes more than 60.000 items) lists performance tends to get really bad.
MS itself suggests that, for lists that are really often accessed, you shouldn’t have more than 2000 items in the same “containers” (that is, a specific folder)
So what can we do when we used to have large lists, with a lot of specific views to help people find what they need, without calling too many items at a time, than, when upgrading to 2010, MS itself tells us : “We can’t assure you that your migration will success if you’ve got more than 5000 items per folder. This is our max limit, please folder your items or delete some of them”
We’re currently working on findong workaround for this, but it seems that we’ll still have to go on foldering some of our lists.
@=8)-DX : Thanks for your Move Item solutions !!! it helped me a lot. This seems to be the only way to keep everything as it is (that is “created”, “created by”, “modified”, “modified by” and GUID !! )
@Keith : Thanks for your post a lot !!
Usefull post……
When I initially left a comment I seem to have clicked the -Notify me
when new comments are added- checkbox and from
now on every time a comment is added I get four emails with
the exact same comment. There has to be an
easy method you can remove me from that service? Appreciate it!
changes ? didnt ? may ? fancy ? employ.
I dugg some of you post as I cogitated they were really
beneficial invaluable
It’s very simple to find out any matter on net as compared to textbooks, as I found this article at this site.
Hi there would you mind sharing which blog platform you’re using?
I’m looking to start my own blog in the near future but I’m having a hard
time deciding between BlogEngine/Wordpress/B2evolution
and Drupal. The reason I ask is because your layout
seems different then most blogs and I’m looking for something
unique. P.S My apologies for getting off-topic but I had to ask!
Thank you very much. It helped me in traversing all the folders and subfolders while pulling a report on site collection level.