Windows SharePoint Services 3.0 (WSSv3) and Microsoft Office SharePoint Server 2007 (MOSS2007) now support the ability to create folders within lists. This is quite frankly one of the coolest features in this release of the products (well, aside from all the OTHER coolest featuresJ), but the point is, it also adds a layer of complexity to how you enumerate lists now if you want to traverse a List folder hierarchy the same way you might want to enumerate a Document Library folder hierarchy.
This is something that was giving me a real headache trying to figure out how to do all day long. (Strangely, the solution I came up with I thought about a couple of days ago, I just didn’t want to have to do it this way, nor did I think I would “Have” to do it this way). I searched around, referenced the SDK, searched around more, but still came up blank. After collaborating with a lot of my peers and friends (And for those of you I pinged on this, thanks for your suggestions), I found that the suggestions they were sending me, were paths I already took and found they would not work J But, just after I harassed them, I realized what the solution was J
What I’m going to do first, is provide you a bit of background on what I attempted, and show you why those solutions don’t workJ. Then show you the solution that will work against both Lists and Document Libraries, or any list in generalJ.
Traversing a Document Library (In WSSv2 or WSSv3) was a simple process. Consider the following code if you were enumerating the document libraries using the lists collection of a SPWeb
// Within some method somewhere
foreach(SPList list in web.Lists)
{
if(list.BaseType == SPBaseType.DocumentLibrary)
{
//Traverse the folder
TraverseFolder(list.RootFolder);
}
}
TraverseFolder might be implemented something like this:
public
long TraverseFolder(SPFolder folder)
{
// Do something interesting on the folder
// Perhaps sum up the file sizes
long lFolderSize = 0;
try
{
foreach (SPFile file in folder.Files)
lFolderSize += file.Length;
if (bRecurseFolders)
{
foreach (SPFolder subfolder in folder.SubFolders)
lFolderSize += TraverseFolder(subfolder);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
return lFolderSize;
}
It would be nice if you could do the same thing with a base SPList, but you can’t J
You may be inclined to start with something like this, the same way I was inclined to do so J
foreach (SPListItem item in list.Items)
{
Console.WriteLine(“List item: “ + item.Name);
Console.WriteLine(“List item type: “ + item.ContentType.Name);
}
// Folders are not counted in list.Items, so we have to look at those seperately.
// But, the Folders collection is just another collection of SPListItems.
foreach (SPListItem item in list.Folders)
{
Console.WriteLine(“List Folder item: “ + item.Name);
Console.WriteLine(“List item type: “ + item.ContentType.Name);
TraverseListItem(item);
}
But there are a couple of problems with this approach
- The items returned from via SPListItemCollection at list.Items, returns ALL items over the whole list hierarchy
- The items returned from via the SPListItemCollection at list.Folders, returns ALL folders over the whole list hierarchy
So consider a list that was structured this way:
- Announcements
- Item 1 in the root of the Announcements list
- Item 2 in the root of the Announcements list
- Folder 1 in the root of the Announcements List
- Item 1 in Folder 1
- Item 2 in Folder 1
- Folder 2 under Folder 1
- Item 1 in Folder 2
- Item 2 in Folder 2
The enumeration of list.Items above would return the following items:
- Item 1 in the root of the Announcements list
- Item 2 in the root of the Announcements list
- Item 1 in Folder 1
- Item 2 in Folder 1
- Item 1 in Folder 2
- Item 2 in Folder 2
And the enumeration of list.Folders above would return the following items:
- Folder 1 in the root of the Announcements List
- Folder 2 under Folder 1
Let’s assume you have not discovered that yet and when you detect that the item is a folder, you try to treat it this way J
You’re TraverseListItem () call may be implemented something like this:
private
static
void TraverseListItem(SPListItem listitem)
{
SPFolder folder = listitem.Folder;
if (folder != null)
{
// This item is a folder, therefore call on contained items recursively.
Console.WriteLine(“Found a folder!!!”);
Console.WriteLine(“Folder: “ + folder.Name);
foreach (SPFile file in folder.Files)
{
// Do something
}
}
}
The problem is that even though the list item is a folder, and you get a valid SPFolder
back from listitem.Folder, but unlike a Document Library, it will have a count of zero.
But hey! There is a listitem.ListItems collection! That would work right?
private
static
void TraverseListItem(SPListItem listitem)
{
SPFolder folder = listitem.Folder;
if (folder != null)
{
// This item is a folder, therefore call on contained items recursively.
Console.WriteLine(“Found a folder!!!”);
Console.WriteLine(“Folder: “ + folder.Name);
foreach (SPListItem item in listitem.ListItems)
{
// Do something
}
}
}
No, that collection returns the base SPListItemCollection from the list itself, just like SPList.Items and this would get you into a never ending loop, as you’d just recurse the same items, over and over, and over, and over, <Hey go away you pesky wabbit!>
Ok, so maybe you rethink it all, and decide to treat the list as a SPDocumentLibrary
and do what you know works. But no, the base type of a List is GenericList, not SPDocumentLibrary, therefore something like this won’t work:
SPDocumentLibrary docLib = (SPDocumentLibrary)web.Lists["Announcements"];
So, just how the heck to you enumerate the folder items in a list when you discover that the item you’re looking at is in fact a folder?
It’s actually very easy, but I would vote it’s a horrible way to have to go about it.
Enter SPQuery:
private
static
void TraverseListFolder(SPFolder folder)
{
// Get the collection of items from this folder
SPQuery qry = new SPQuery();
qry.Folder = folder;
SPWeb web = null;
try
{
web = folder.ParentWeb;
SPListItemCollection ic = web.Lists[folder.ParentListId].GetItems(qry);
foreach (SPListItem subitem in ic)
{
Console.WriteLine(“List item: “ + subitem.Name);
Console.WriteLine(“List item type: “ + subitem.ContentType.Name);
if (subitem.Folder != null)
TraverseListFolder(subitem.Folder);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
// Cleanup that nasty web reference.
if (web != null)
web.Dispose();
}
}
Ok so cool, we now have a way to get ONLY the items in a folder within a list at any level without introducing any items that are not in scope.
So, we’ll just use this method, and to traverse the list, but hey, where do you start? Well, fortunately SPList exposes a RootFolder property that gives you a SPFolder object to start with J
So we can write our List traversal Kick off method as such:
private
static
void TraverseList(SPList list)
{
Console.WriteLine(“Traversing list: “ + list.Title);
Console.WriteLine(“Base Type: “ + list.BaseType.ToString());
TraverseListFolder(list.RootFolder);
}
Simply pass in a reference to a list, and away we go. Very compact, and it works regardless if the list being passed in is a Document Library or any kind of list.
Hope this helps!
- Keith Richie
I wonder what the cost of querying/requerying the list for each folder/subfolder will cost just to iterate what should have been simple to hierarchy to traverse. Yuck!
We need to introduce Joe Celko’s concepts like nested sets for hierarchies (http://www.developersdex.com/gurus/articles/112.asp) to the Microsoft product teams.
Comment by Todd Bleeker — January 30, 2007 @ 8:13 am
I agree… I too am wondering what the cost is going to be. I won’t argue that it’s probably not the most performant.
Considering that a SPFolder.Files reference is going to make a trip to the database, and the call to web.Lists[folder.ParentListId].GetItems(qry); is goint to make a trip to the database, It boils down to running a SQL profile trace to see how chatty one way or the other is.
That would give us a comparison for “Document Libraries”, but not for lists :).
Comment by Keith Richie — January 30, 2007 @ 9:06 am
Thanks for the detailed explanation. Another vexing question is - how do you move list items from one folder to another. There is no way (that I can find anyway) to do it in web UI. There is no Explorer view option available for the Lists like you have with Doc Libs. On publishing sites, you can go into the Site Manager page and select list items and choose to Move them, but this is not available to the average user.
As for via the OM, the only way I’ve come up with so far is to create a copy of the list item at the new location, and then delete the original list item. Of course when you do this you lose the original item’s Creator, Modifier, Created, Modified values, which IMHO is wrong.
SPListItem doesn’t seem to have the expected/needed MoveTo() method.
The Folder and Url properties are read-only.
I’ll appreciate to know if anyone figured out a better approach.
Comment by Larry Kuhn — February 13, 2007 @ 11:32 pm
Now that’s a good one :), and not one I have an immediate answer for
So SPListItem.Copyxxx() doesn’t preserve the Creator,Modifier, etc. Well I can see why it wouldn’t do that, as it’s a “Copy” and I’m sure that the copied item now shows the user context information that your code is running under right?
What about getting a folder reference to the folder containing the item and call SPFolder.MoveTo() ? Would that help?
Of course, that means that it’s all or nothing for the folder, and I would still see a need for individual item moves.
Comment by Keith Richie — February 14, 2007 @ 7:23 am
[...] Traversing SharePoint List Folder Hierarchies MOSS 2007 now support the ability to create folders within lists. This is quite frankly one of the coolest features in this release of the products (well, aside from all the OTHER coolest featuresJ), but the point is, it also adds a layer of complexity to how you enumerate lists now if you want to traverse a List folder hierarchy the same way you might want to enumerate a Document Library folder hierarchy. [...]
Pingback by Quick Links Krichie Edition » Sharepoint BUZZ - Blogging the Best of SharePoint 2007 (MOSS) — February 26, 2007 @ 6:21 am
1.) What are the command line args for the following Sharepoint code
“Sample code to either Calculate size of all files or Export Files from a web”
I think its url -export some_third_arg???
2.) How do I install the program after I have built the exe file?
This is the URL that I am referring too.
http://blogs.msdn.com/krichie/archive/2005/02/15/373271.aspx
Thnaks,
Dale
Comment by Dale — March 10, 2007 @ 1:45 am
Hi,
SPList.Folders property will not return a colection containing all the folders and subfolders in that list?
Comment by Florin Muntean — April 7, 2007 @ 10:48 pm
Yes it will, that’s exactly what it does, and what I refer to in this post when I state:
“…the enumeration of list.Folders above would return the following items:
Folder 1 in the root of the Announcements List
Folder 2 under Folder 1
”
I gives you back a flat set of the folders within the list or document library, but does not give you heirarchy nor show containment.
Comment by Keith Richie — April 8, 2007 @ 1:12 am
I was using the SPList.Folders property but I got some issues with lists that were imported from list templates and have custom folder content types.
The folders are not apearing into the Folders collection until I edit them manualy in SharePoint.
Any ideea why?
Comment by Florin Muntean — April 8, 2007 @ 2:16 am
It could be (And this is just a guess) that when you imported them, you imported them but they were not checked out, and the user account being used to enumerate SPList.Folders is only seeing the checked in versions.
I suspect that when you edit them manually, they are being checked in, and then discoverable?
Comment by Keith Richie — April 8, 2007 @ 7:33 pm
hi,
Is there any webservice method to get items from list recursively from folders and subfolders at oneshot.
thanks,
surya
Comment by suryakiran — May 4, 2007 @ 1:10 pm
[...] As you all might or might not know, WSS 3.0 and MOSS 2007 support folders in lists. This is honestly a really cool feature/enhancement to the list functionality. This opens the door to a lot more possibilities.However, you need to be a little careful with traversing lists that you create folder heirarchies in. Traversing a list is not as obvious as you think. I was a little surprised that the solution was not intuitive.Here are some of the caveats that I discovered:1. A call to list.Items will not bring forth your folders. Instead, it will bring forth ALL the regular items, regardless of the folder heirarchy they live in. So it doesnt matter if an item is at the root level or 4 levels deep - it will show up in the call to list.Items.2. We have another property available for the folders :). However, a call to list.Folders will bring forth all the folders regardless of their heirarchy in the list.Keith Ritchie has a solution that works and he has done good homework around this problem. Be sure to have a look at his post here. [...]
Pingback by Faraz Khan's Sharepoint blog: Traversing lists in MOSS 2007 — September 22, 2007 @ 2:38 am
Hi I am having an issue with the explorer view: hoping you can explain it.
Any document loaded had a created and modified date, in the All Items View - if you modify the doc it changed the modified date, but in the explorer view when you right click and say properties, the created on and modified on date stays the same. Can you explain why to me please?
Comment by Louise van der Bijl — October 10, 2007 @ 5:36 pm
Has anyone come up with a way to copy or move list folders along with their content? I have tried and failed so far. When ever I call the MoveTo method on a SPFolder I get the exception: “Folder //destination/source folder name// does not exist”
I would think that the source folder would be created there as the destination folder without instruction.
Can anyone help me out here?
Comment by Esther — November 27, 2007 @ 9:12 am
Has anyone figured out how to traverse the list BACKWARDS? For example, if I have a reference to a particular folder, how can I traverse the list back up to the root folder? I’ve searched everywhere and not found any mention of this. Any help would be greatly appreciated!
Comment by Bitwise — November 28, 2007 @ 12:32 am
Have you tried walking the SPFolder.ParentFolder property until it is null?
Comment by Keith Richie — November 29, 2007 @ 2:57 am
If the point is just to query items in a library recursively (without folders), why not just use GetItems and set the Scope attribute to recursive in your SPQuery.ViewAttributes property?
myQuery.ViewAttributes = “Scope=\”Recursive\”";
This seems tremendously more efficient than looping.
Comment by Peter Brunone — February 6, 2008 @ 9:54 pm
That would probably work, but I needed to ensure that I only retrieved the immediate children of a given level, rather than the entire batch of items .
I’m sure I could do this, then do some logic to parse it out after the fact, but this solved my problem at the time. Thanks for the great suggestion though!
Comment by Keith Richie — February 6, 2008 @ 10:14 pm
Thanks for this article. Obviously you put a lot of work into it.
Comment by Kevin — February 12, 2008 @ 10:41 pm
I’ve got a couple of things I’ve used:
SPFolder.SubFolders: for a list this returns the correct subfolders of a given folder, starting with SPList.RooFolder, even in a custom list.
Moving SPListItem objects with SPFile.MoveTo - use this to move SPListItems.. It seems jerky, because logically a simple SPList - SPListItem object is not a file.. but actually it works. Nevertheless there is a problem - I think SPListItem objects return null for .File. So you can get around this with:
SPFile item_file = dm.thisWeb.GetFile(dm.thisItem.UniqueId);
item_file.MoveTo(NEWURL);
… but you have to do a GetFile() for each listitem. The whole thing just shows how stupid it is to implement a folder structure for objects and then leave out simple folder-related tasks like moving, listing, etc. (or the nonexistant SPListItem.ParentFolder object..) direct simple folder queries to the Database would be much more optimal.
Basically with a simple SPList you use SPList.RootFolder and SPFolder.SubFolders to traverse the list, but then only display a single folder’s content using the query.
Another way to get all list items within a certain SPFolder hierarchy is to query with MyFolderServerRelativeUrl</BeginsWith or some such query (there are quite a few ListItem properties cotnaining the URL) or to compare the SPListItem.Url property with your folder url -
SPListItem.Url.Substring(0, SPListItem.Url.LastIndexOf(”/”)) == myFolder.ServerRelativeUrl.. or similar.
Why do we have to FIND OUT and GUESS how SharePoint works.. pity they didn’t just make it work intuitively and document it properly from the start.
Comment by =8)-DX — February 19, 2008 @ 2:31 am
Pinging back.
Thanks for this.
Comment by Garth — March 25, 2008 @ 3:35 pm
Excellent write up. Will be very helpful in what I am working on.
I have a semi-related question. Being new to SP, I noticed that when I use foreach to enumerate over the Items in a List, that I get huge memory bloat that does not release. Is there a way around this? What is causing it?
Any help is appreciated.
Comment by DanMc — March 26, 2008 @ 9:17 am
Not sure the direct calls your making during the enumeration, but if you are accessing the lists ParentWeb, it will instantiate a new SPWeb instance that you should call dispose on.
See: “Best Practices: Using Disposable Windows SharePoint Services Objects”
http://msdn2.microsoft.com/en-us/library/aa973248.aspx
Article was written for WSS 2.0/SPS2003 but still applies to WSS 3.0/MOSS
Comment by Keith Richie — March 26, 2008 @ 6:57 pm
Keith, thanks for the quick reply. I will take a look at the article, but I am already properly disposing of my Web and Site references.
Is there a way we can take this conversation “off-line” so as to not clutter up your blog?
Comment by DanMc — March 26, 2008 @ 11:13 pm
Keith, is it possible to add a property to a list folder? I’m asking this because I need to show the items and folders of a Sharepoint list in an application outside Sharepoint. They have to be shown in a specific order, something like this:
Item #1
Folder #1
Subitem #1
Subitem #2
Subitem #3
Item #2
Item #3
Folder #2
Subitem #1
Subitem #2
Item #4
Item #5
So I thought of solving this by using a property (or something of the sort) in the folder, just like a column in the list item, to store the display sequence number.
How can I do that?
Comment by Claudia — April 8, 2008 @ 12:38 am
I believe you can. You just need to create a new content type based on the folder content type, and add the columns that you require to the folder.
Comment by Keith Richie — April 18, 2008 @ 12:23 am
DanMc:
>>Being new to SP, I noticed that when I use foreach to enumerate over the Items in a List, that I get huge memory bloat that does not release.
If you use the SPList.Items collection it will theoretically cache ALL the items in the list in memory (SPList objects). In lists with thousands of items this is not optimal, and in large lists it could lead to memory clutter. Much better is to use Views and Queries to return only the required items. I think you can divide the query into batches by setting the SPQuery.ListItemCollectionPosition and SPQuery.RowLimit properties. Basically it all boild down to using the Items collection as little as possible. us ItemCount to get the number of items.
Claudia:
>>is it possible to add a property to a list folder?
Each SPFolder object in a custom list has a corresponding Item object accessible through SPFolder.Item or SPList.GetItemByUniqueId(SPFolder.Item). You can get the folder’s properties using this SPListItem object just as you would with any other list item.
Comment by =8)-DX — April 18, 2008 @ 3:49 pm
[...] Nested Root Folder Batman! In one of my previous posts, I talked about how I was using the SPQuery object to traverse SharePoint List/Folder [...]
Pingback by Holy Nested Root Folder Batman! « Krichie - That SharePoint Guy — April 22, 2008 @ 12:52 am
I have folders around 3000 in my shared document library & it seems very slow when we use via webservice…
how can we tune up this folders in sharepoint to be fast response.
performance is needed right now!
i had created webservice to get site defention which retrives all folders & path enumerating.
Comment by PrashanthSpark — November 21, 2008 @ 10:41 am
is that 3000 folders at the same level?
Comment by Keith Richie — November 24, 2008 @ 9:10 pm