July 14, 2013

Exist v.s Existing and Auto Exist in MDX – Part 2–Usual confusion

Please read the Part1 of the same series to get the continuity

Usual confusions

  • Exists and “Auto exists” are nearly same only difference is Exists function don’t display the second set .

Actually no. One set the context other doesn’t

SELECT [Measures].[Reseller Sales Amount] ON 0           
, Exists( [Product].[Subcategory].[Subcategory].Members
,[Product].[Category].[Clothing]
)on 1
FROM [Adventure Works]
where [Product].[Category].[Bikes]

Above query will produce no results because we are slicing by two different members(Bikes & Clothing) at the same time. But the the query allows you to use the same dimension hierarchy in both rows Axis and slicer axis.


But the below query will produce errors, because it sets the context


SELECT [Measures].[Reseller Sales Amount] ON 0      
, [Product].[Subcategory].[Subcategory].Members *
[Product].[Category].[Clothing]

on 1
FROM [Adventure Works]
where [Product].[Category].&[1]

image_thumb[22]



  • Existing keyword is just an extension of Exists, we can achieve the similar results with exist and currentmember function



Yes, But use of existing will improve the maintainability of code


Both the below codes will produce the same results


with member countofsubcategory as 
count( existing [Product].[Subcategory].[Subcategory].Members)
SELECT {[Measures].[Reseller Sales Amount], countofsubcategory} ON 0
, [Product].[Category].[Category].Members
on 1
from [Adventure Works]

with member countofsubcategory as
count( exists( [Product].[Subcategory].[Subcategory].Members
,[Product].[Category].currentmember ))
SELECT {[Measures].[Reseller Sales Amount], countofsubcategory} ON 0
, [Product].[Category].[Category].Members
on 1
from [Adventure Works]
image



  • DO we have not exists function ?



No. But this can be simulated with except function


The below query produces 8 rows of data


SELECT [Measures].[Reseller Sales Amount] ON 0      
, Exists( [Product].[Subcategory].[Subcategory].Members ,
[Product].[Category].[Clothing]
)

on 1
FROM [Adventure Works]
image 


With the except function in the below query it produces the opposite set with 30 rows 


SELECT [Measures].[Reseller Sales Amount] ON 0      
,EXCEPT([Product].[Subcategory].[Subcategory].Members,
Exists( [Product].[Subcategory].[Subcategory].Members ,
[Product].[Category].[Clothing]
)
)
on 1
FROM [Adventure Works]

image_thumb[24]

image_thumb[26]



  • Generate function forces it’s own context against existing keyword



Check the below query. It is clear that existing keyword uses the generate function’s context and that is why we got 3 subcategory against all category in the result set

with member countofsubcategory as 
Generate ([Product].[Category].[Bikes]
, count( existing [Product].[Subcategory].[Subcategory].Members)
)
SELECT {[Measures].[Reseller Sales Amount], countofsubcategory} ON 0
, [Product].[Category].[Category].Members
on 1
from [Adventure Works]

image_thumb[28]

 

This is an another proof that existing keyword is simply an extension of Exists function

Happy Reading Smile

July 7, 2013

Exist v.s Existing and Auto Exist in MDX – Part 1

Though Exist, existing and Auto exist are conceptually very simple, they are the root cause for many confusion in the complex MDX queries. So let us do a simple comparisons

 

Normal Behaviour

Auto Exists

Exists

Existing
Theory

When the two sets are cross joined it will produce all possible combination. E.g. if we join two sets with 4 dimension each then it will produce 16 members

It forces the natural hierarchy in the below Scenarios


When two sets containing members of the same dimension are cross-joined, the resulting set is limited to those combinations of members actually observed in that dimension.
When the WHERE
clause contains a member of a dimension, sets along the axes containing members from that same dimension are limited as well.

It forces the natural hierarchy with out returning the forcing/second set

Auto exist is not applicable to calculated members and if we want to force this behaviour then we need existing keyword

When to useWhen we want to display the measures based on two different dimensionWhen we want to display the measures based on the members in different level from same hierarchyAchieve the same Auto exists results with out displaying second DimensionForces the local context in the calculated members

Normal Behaviour

Run the below query

SELECT [Measures].[Reseller Sales Amount] ON 0
, [Product].[Category].[Category].Members on 1
from [Adventure Works]

image

Based on the above query we have 4 Categories in the Product Categories Hierarchy

SELECT [Measures].[Reseller Sales Amount] ON 0
, [Geography].[Geography].[Country].members
on 1
from [Adventure Works]
image

Based on the above query we have 6 countries under geography hierarchy

Let us Cross Join it

SELECT [Measures].[Reseller Sales Amount] ON 0
,[Product].[Category].[Category].Members
* {[Geography].[Geography].[Country].Members
} on 1
from [Adventure Works]

image

image

So along with the header row it has returned 25 rows which is direct cross product of 4 Products and 6 countries (24 data row + 1 Header row)

What other ways to achieve the cross join

You can use the cross join function as shown below

SELECT [Measures].[Reseller Sales Amount] ON 0
,crossjoin ([Product].[Category].[Category].Members
, [Geography].[Geography].[Country].Members
) on 1
from [Adventure Works]

Even if you remove the Crossjoin keyword in the above query,  it will still produce the same result .

 
SELECT [Measures].[Reseller Sales Amount] ON 0
      ,([Product].[Category].[Category].Members
, [Geography].[Geography].[Country].Members
         ) on 1
from [Adventure Works]
image
 The reason is MDX will try to resolve the other dimensions of the cube for every combination (remember the partial tuple law)and will get you the cross join.

Auto Exists

In the above example we have used the cross join between two different hierarchies (Product.category and geography.country). let us join between category and sub category from same product hierarchy.

Step 1: Let us find the count of subcategory first

SELECT [Measures].[Reseller Sales Amount] ON 0
, [Product].[SubCategory].[SubCategory].Members on 1
from [Adventure Works]
image
 Along with header row we have 38 subcategories, let us cross join subcategory and category.

 

SELECT [Measures].[Reseller Sales Amount] ON 0
, [Product].[Category].[Category].Members *
[Product].[Subcategory].[Subcategory].Members on 1
from [Adventure Works]
image
image

As you can see from the output though it has given the cross product, it still only has the same 38 (37 rows of data + 1 header) rows in the output. as per the normal behaviour it should be 4*38 = 152

The reason is, based on the Product category hierarchy, only valid category and Subcategory combinations are produced. 

 

What is Existing 

Let us create a calculated member in the above query
with member countofsubcategory as 
count( [Product].[Subcategory].[Subcategory].Members)
SELECT {[Measures].[Reseller Sales Amount], countofsubcategory} ON 0
, [Product].[Category].[Category].Members *
[Product].[Subcategory].[Subcategory].Members
on 1
from [Adventure Works]
image 
Though we are displaying each subcategory in the list, still we got the total count . This is because Auto exist doesn’t control the calculated members  
 
So by inserting Existing keyword in the above query,it will produce the expected result which 1 per subcategory
with member countofsubcategory as 
count( existing [Product].[Subcategory].[Subcategory].Members)
SELECT {[Measures].[Reseller Sales Amount], countofsubcategory} ON 0
, [Product].[Category].[Category].Members *
[Product].[Subcategory].[Subcategory].Members
on 1
from [Adventure Works]
image

 

What is Exists function ?

Exists uses the “Auto exist” functionality but avoid displaying the “second set” in the results

SELECT [Measures].[Reseller Sales Amount] ON 0      
, Exists( [Product].[Subcategory].[Subcategory].Members ,
[Product].[Category].[Clothing]
)
on 1
FROM [Adventure Works]

image

It only displays the subcategories which are related to Clothing category without displaying category in the result.

June 25, 2013

Step By step Guide to integrate SQL management studio with Subversion

 
If you have worked in .net and migrated to database like me then you will surely miss the free add-ins for subversion (ankhsvn). As more and more companies are moving towards the open source versioning system like subversion, it is really painful that Microsoft SSMS studio doesn't have any add-in for subversion.
 
There is a neat and simple solution available from Redgate but it requires license for each PC so if you are working in a place where purchase decisions are difficult to come by then this solution can be very handy.
 
Required: TortoiseSVN in your local machine and write access to subversion repository
 
Create or Check out an existing solution
                                               
·        Decide and create a working folder in C: (You can use other drives but Avoid Network drives)
<!--[if !supportLists]-->o   <!--[endif]-->Ex.  “C:\Test Project”
·         Create a folder in the Subversion repository 
               
 

<!--[if !supportLists]-->·        Right click on the newly created folder and Integrate it with your work folder using the checkout option
  

Once you checked in, your folder will appear like below
 
 
<!--[if !supportLists]-->                                                 Note: You can also check out an existing solution to your work folder
 
·        Create a project in the SQL management Studio and store the solution into above working directory (c:\testproject

 
 
 
Create External Tools
  Open the external tools window
 
 
 

Create External Tools
  Enter the title, command , Arguments and initial Directory as per the below list

 
 
List of commands and parameters
 
Commit:
 
Title : SVN CommitSollution
Command : C:\Program Files\TortoiseSVN\bin\TortoiseProc.exe
Arguments : /command:commit /path:"$(SolutionDir)"
Intial Directory: $(SolutionDir)
 
CommitFile:
 
Title : SVN Commitfile
Command : C:\Program Files\TortoiseSVN\bin\TortoiseProc.exe
Arguments : /command:commit /path:"$(ItemFileName)$(ItemExt) "
Intial Directory: $(ItemDir)
 
Revision History
 
Title : SVN-RevisionHistoryforItem
Command : C:\Program Files\TortoiseSVN\bin\TortoiseProc.exe
Arguments : /command:log /path:"$(SolutionDir)"
Intial Directory: $(SolutionDir)
 
 
How To use ?
 
Choose the projects and select CommitSollution from your tools
 
 


 
 
It will list out all the solution files. Please choose the new Project(s) which you would like to add.
 
Note: You can use the All button to select all the files automatically
 
 



Commit changes of your files

If you prefer to commit the changes in a single file then “CommitFile” option can be used
<!--[if !supportLists]-->a.       <!--[endif]-->Select the file which has had some change from the last check in
<!--[if !supportLists]-->b.      <!--[endif]-->Only the file which has had any change will appear
 


 

Check the revision History

Another common requirement and advantage of using source control would be comparing it against one of the older versions.
 
In my example I have two versions:
First one I checked in during the initial upload and the upload in the previous step.
In the second version I have added the below query
select * from [dbo].[DimEmployee]” into the sampleQuery
 
If I select the samplequery.sql and select SVN-Revisiohistoryforitem then it will appear as below
 

 
 
You can select the version of your choice and compare it with working copy.
 




Just works like a charm J
 

Thanks to John Rummell on external tools and SSMS
 

May 12, 2013

Use SSMS 2012 with SQL server 2008




SQL server Management studio 2012 is freely downloadable from Microsoft site and it is compatible with Sql server 2008.I was using this for past 3 months and found it very useful, so I listed down some useful new features and changes

Even if your company is not ready to upgrade to 2012, this free client upgrade is worth the try

Some useful features

  1. Usability : SSMS 2012 is powered by visual studio so the UI looks lot similar with Visual studio  
  2. I always have issues with 2008 intellisense (autofill) which is very slow and sometimes even manual refresh take longer time to refresh the metadata. But 2012 is very impressive and i am much surprised about the meta data management.
  3. In 2012 intellisense accept part name search.  E.g . If your table name is abc_xyz_ETL then you can type xyz which will pick all the tables which contains the xyz while writing the query
  4. The new improved MDX query editor with MDX intelisense is very useful for SSAS query editing. 
  5. New Source safe integration - the SSMS 2012 can be easily integrated with team foundation server with a free plugin. I am really hoping to see some free plugin for subversion soon. 
  6. Projects are easy to create and maintain
  7. You can create custom template which can be created and used across organisation
  8. New SQL CMD Mode. So you can mix DOS and sql command in the script. Very useful to move and import from excel/csv files via DB links
  9. Very easy to pass the list of servers to the new joiner as you can export the registered servers to other machine

 

For more information please refer to

April 20, 2013

Convert rows into comma separated column using single query with XML

Convert rows into comma separated column using single query with XML

Have you ever try to covert list of rows into an comma separated column? There are many solution available based on the coalesce but I found an interesting fact about the XML Path by which we  can easily achieve the same functionality without much of hassle.

In the below example I have created a query to extract table and column names from the information_schema  for two test tables

SELECT table_name, 
       column_name 
FROM   information_schema.columns 
WHERE  table_name IN ( 'DimProductCategory', 'DimCurrency' ) 
 
output

image

Now if we just want to display two rows for each table and concatenate the columns into single value with a delimiter then we can simply achieve this by below query.

SELECT Distinct col2.table_name, 
       Stuff((SELECT ',' + column_name
              -- Stuff used here only to strip the first character which is comma (,). 
              FROM   information_schema.columns col1 
              WHERE  col1.table_name = col2.table_name 
              FOR xml path ('')), 1, 1, '') 
FROM   information_schema.columns col2 
WHERE  table_name IN ( 'DimProductCategory', 'DimCurrency' ) 


image

Further Learning

March 10, 2013

SSRS lookup and lookup set – Similar to Excel


On the other day I came across a interesting functionality in the SSRS. They are Lookup and Lookup set functions , these functions are very useful for multi source reports
Simple Abstract :
Lookup : Just like the Excel Vlookup function which can get the data from another dataset
Lookupset: Same like lookup but returns all the matching results and will be very useful for displaying all the matching results, This can be converted to comma separated form using Join function.
Check the Below Blog for details
http://www.bidn.com/blogs/DustinRyan/bidn-blog/2037/lookup-and-lookupset-functions-new-in-ssrs-2008-r2
 

Technorati Tags:

February 10, 2013

Import Active Directory (AD) Groups and user names via SSIS into SQL server table

Before getting into the technical details, I will explain the need for this.
  1. If you manage SQL server Security via AD Groups due to high volume of business users then you would have already hit the problem and looking for solution to import the mapping details of groups vs users
  2. If you are using SSRS suit for the reporting needs and if you have large user base then AD group based security is the best way to segregate user access. But once again to answer who has what rights you need this mapping information.

Step 1: Create the table structure necessary to import the SSIS Groups and Names in two tables

Table 1: ActiveDirectoryUserGroups – This tables gives you the relationship between users and their group name they belongs to
CREATE TABLE [dbo].[ActiveDirectoryUserGroups](

[id] [int] IDENTITY(1,1) NOT NULL,

[GroupName] [varchar](1000) NULL,

[username] [varchar](1000) NULL

)  

 

GO



Table 2:ActiveDirectoryGroupRelation – This table gives you the relationship between Groups


CREATE TABLE [dbo].[ActiveDirectoryGroupRelation](

[id] [int] IDENTITY(1,1) NOT NULL,

[GroupName] [varchar](1000) NULL,

[ParentGroupName] [varchar](1000) NULL

) 

 

GO

 


Step2: Create the necessary connection to your database where the above table exist

image 

Step 3: Create SQL task to clear the data before load.

(Alternatively, you can import the changes alone via lookup or in other words use the incremental load approach . But that is not the point of this blog so to keep this simple I will just truncate the records before load)
image 

Step 4: Create a data flow task in the control flow to fetch mapping between groups

(Name it suitably  E.g “DFL_ActiveDirectory_Group_relation_load”)
Create Three Variables in the data flow task scope
2jpmvrye

Note: Replace Domain with your mail Domain. E.g. “Pinnacle-Int” and “Extension” with your Domain extension E.g. “Com”

Create a “Source Script Component”. as shown below.

Make sure to declare all three variables as read only variables as shown below
Choose your choice of scripting language

image 
Set the outputs correctly
image
Paste the below code in the Scripter



Public Overrides Sub CreateNewOutputRows()

 

        Dim de As New DirectoryEntry

 

        Dim searcher As New DirectorySearcher

        Dim search_result As SearchResultCollection

        Dim result As SearchResult

        Dim props As ResultPropertyCollection

        Dim MemberOfList As StringBuilder

        Dim values As ResultPropertyValueCollection

        Dim name As String

        Dim groups As ArrayList

 

        Using (de)

 

 

            de.Path = Me.ReadOnlyVariables("Path").Value.ToString

 

 

            Using (searcher)

 

                searcher.SearchRoot = de

 

                searcher.Filter = Me.ReadOnlyVariables("Filter").Value.ToString

                searcher.SearchScope = SearchScope.Subtree

 

 

                ' Retrive ActiveDirectory column list

                searcher.PropertiesToLoad.Add("samaccountname")

                searcher.PropertiesToLoad.Add("memberof")

 

                'sort the result

                searcher.Sort = New SortOption("samaccountname", SortDirection.Ascending)

 

 

                ' you can set your own limits

                searcher.PageSize = Me.ReadOnlyVariables("MaxRecord").Value

 

 

 

                'Retrieve the results from Active Directory

                search_result = searcher.FindAll()

 

 

                MemberOfList = New StringBuilder 'Members/Groups list.

          

                Dim entry As DictionaryEntry

 

                For Each result In search_result

                    props = result.Properties

 

 

 

                    For Each entry In props

 

                        values = entry.Value

 

                        If entry.Key.ToString = "samaccountname" Then

                            name = GetSingleValue(values)

                        End If

 

                        If entry.Key.ToString = "memberof" Then

                            groups = GetGroups(values)

 

                            For Each group In groups

                                ListofusersoutputBuffer.AddRow()

 

                                ListofusersoutputBuffer.Name = name

 

                                ListofusersoutputBuffer.MemberOf = group

                            Next

 

                        End If

 

                    Next

 

                Next

 

            End Using

 

        End Using

 

 

 

    End Sub

 

    Private Function GetSingleValue(ByVal values As ResultPropertyValueCollection) As String

 

        For Each val As Object In values

 

            Return val.ToString()

        Next

 

        Return Nothing

    End Function

    Private Function GetGroups(ByVal values As ResultPropertyValueCollection) As ArrayList

 

        Dim valueList As New ArrayList()

        For Each val As Object In values

 

            Dim memberof As String = val.ToString()

 

            Dim pairs() As String = memberof.Split(",")

            Dim group() As String = pairs(0).Split("=")

 

            valueList.Add(group(1))

        Next

 

        Return valueList

    End Function



Connect the Appropriate Destination as shown below
image
Do the Appropriate mappings for ActiveDirectoryGroupRelation
image
Test whether the load is successful



Step 5: Create a data flow task in the control flow to fetch mapping between user and groups

(name it suitably E.g “DFL_ActiveDirectory_user_group_relation_load”)
Follow the same similar steps as above. Only difference is load them with below Variables.
Create Three Variables in the data flow task scope
m4fmfsat
Note: Replace Domain with your mail Domain name. E.g. “Pinnacle-Int”  and “Extension” with your Domain extension E.g. “Com”
Also make sure to load “ActiveDirectoryUserGroups” table this time
Your Final solution should look like below
image
Hope that is helpful!