Showing posts with label scenario. Show all posts
Showing posts with label scenario. Show all posts

Sunday, March 25, 2012

Advice needed

I’d appreciate an advice about the following data mining scenario. A bank institution has customers and keeps tracks of the customer demographics data (e.g. income, age, etc). A customer could have purchased one or more products (checking, savings, CD, and other accounts). My task is to predict the likelihood for a customer to purchase a new product. For example, if the custom has checking and savings accounts, what’s the likelihood for the customer to purchase a CD account so we can offer this product do the customer.

My questions:

  1. What data mining task (segmentation, classification, etc) does this requirement represent?

  2. What algorithm(s) is best suited for this task?

  3. For a given customer, will I be able to predict the buy probability for all products or do I need to query the mining model to predict one product at the time?
Shrek banks are great candidates to leverage product recommendations. You are on the right path. A few quick pointers to your questions:

1. It makes sense to first utilize MS Clustering see learn more about what customers have in common. This will help determine what key drivers (e.g. geography, demographic, socioeconomic, tenure, etc.) are more important that others in recommending products.

2. MS Decision Trees and Neural Nets will do a great job of identifying what "x" product to offer to "y" customer. Each alogirithm has its own benefits.

3. If you properly set up the model you can set the output to deliver a scored list of product recommendations for each customer record instead of one product at a time. Remember the point of a product recommendation model is to provide multiple products that a customer may be likely to sign up for (think Amazon.com they offer multiple recommendations not just one at a time). For example, the models we have developed for banks will predict a cross-sell product (Mortgage), an up-sell product (gold membership), an add-on (checking plus) to each scored customer record.

Hope this helps.

Jeff

Please visit Apollo Data Technologies http://apollodatatech.com/|||

Jeff,

Thank you so much. I will definately try out your recommendations. If you don't mind, I would appreciate some additional pointers. Our system captures the customer products (checking, savings, CD, etc), account balances, profitability factors, and lmited demographics data (address and age). A big concern I have is that that input parameters may not be sufficient for a good cross-selling prediction.

1. Based on your experience with the banking industry, what could be the most important factors that may influence the customer (individual or business) to buy a new product?

2. Are there any companies that could provide tha missing important inputs (demographics and financial) if our system and the bank cannot supply them? Any idea about how much such services can cost?

I appreciate your help.

|||Shrek - good questions...

1. Your first question on key drivers is the holy grail. The key drivers really vary based on many factors - is it a national or regional bank, full-service, strong online presence, how many branches, etc? What I can say confidently without knowing the above answers and looking at the data is that income level and geo/demo are often key drivers. The best way to accurately determine this is to build a clustering model utilizing the MS Clustering algorithm.

2. Yes - there are many companies that are more than happy to sell you geo/demographic and socioeconomic data (at the individual level, household, business) you can overlay ontop of your customer records to improve the richness of the data set. The price ranges based on how many records and overlay attributes you determine you need. We have built a service around identifying which 3rd party data will be the best predictor for identifying product recommendations (these vendors will provide a free sample for evaluation). As a heads up this service pays for itself b/c often times the 3rd party data sold is of poor quality; however, our process identifies the best data sets available and the minimal amount of data needed to purchase so you are not over-buying data that is of little use.

Feel free to contact me if you need some additional pointers info@.apollodatatech.com

Thursday, March 22, 2012

AdventureWorks DB - Description/Scenario

Hi,
does anybody know where I can find a printable description of the
AdventureWorks DB. I mean the explanation in english (or french) of this DB:
what is it about? Simple description of the architecture? ...Etc...
Thanks,
ChrisSQL Server Books Online has numerous topics on AdventureWorks including
topics that describe each table. Just type in AdventureWorks in the Index
of Books Online to get started. If you don't have Books Online installed,
you can download the latest update from here:
http://www.microsoft.com/technet/pr...oads/books.mspx
You may also want download the database diagram of AdventureWorks from here:
http://www.microsoft.com/downloads/...&displaylang=en
Gail Erickson [MS]
SQL Server Documentation Team
This posting is provided "AS IS" with no warranties, and confers no rights
"Club Internet" <c.le_roq@.caramail.com> wrote in message
news:4456448e$0$296$7a628cd7@.news.club-internet.fr...
> Hi,
> does anybody know where I can find a printable description of the
> AdventureWorks DB. I mean the explanation in english (or french) of this
> DB: what is it about? Simple description of the architecture? ...Etc...
> Thanks,
> Chris
>sql

AdventureWorks DB - Description/Scenario

Hi,
does anybody know where I can find a printable description of the
AdventureWorks DB. I mean the explanation in english (or french) of this DB:
what is it about? Simple description of the architecture? ...Etc...
Thanks,
ChrisSQL Server Books Online has numerous topics on AdventureWorks including
topics that describe each table. Just type in AdventureWorks in the Index
of Books Online to get started. If you don't have Books Online installed,
you can download the latest update from here:
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
You may also want download the database diagram of AdventureWorks from here:
http://www.microsoft.com/downloads/details.aspx?familyid=0F6E0BCF-A1B5-4760-8D79-67970F93D5FF&displaylang=en
--
Gail Erickson [MS]
SQL Server Documentation Team
This posting is provided "AS IS" with no warranties, and confers no rights
"Club Internet" <c.le_roq@.caramail.com> wrote in message
news:4456448e$0$296$7a628cd7@.news.club-internet.fr...
> Hi,
> does anybody know where I can find a printable description of the
> AdventureWorks DB. I mean the explanation in english (or french) of this
> DB: what is it about? Simple description of the architecture? ...Etc...
> Thanks,
> Chris
>

Tuesday, March 20, 2012

Advantages?

I would welcome opinions on whether FTS would help in our scenario.
The table has about 28 million rows and has about 10,000 rows added every
morning in an intensive batch process. Then another batch query process is
run that looks for partial name matches.
Eight of the fields we query against are char or varchar, all with single
words.
Sample query: SELECT casenumber FROM table WHERE fname LIKE 'j%' AND lname
LIKE 'smi%'. Rreturns 55,874 rows in 7 seconds in QA.
I am also concerned about time to populate FT index every day.
Machine is dual Xeon 3.2 with 4GB ram, OS on RAID1 & SQL data on RAID5. Win
2003 Server and SQL2000, with SQL2005 later this year.
Thanks much!
Wow! That's fantastic speed for a like. I'd try change tracking. IIRC on a
similar machine I was able to hit 2000 rows per minute with SQL 2000.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Bil Click" <BilClick@.discussions.microsoft.com> wrote in message
news:FFF888F6-8310-4B1F-83B0-399830C17F53@.microsoft.com...
>I would welcome opinions on whether FTS would help in our scenario.
> The table has about 28 million rows and has about 10,000 rows added every
> morning in an intensive batch process. Then another batch query process is
> run that looks for partial name matches.
> Eight of the fields we query against are char or varchar, all with single
> words.
> Sample query: SELECT casenumber FROM table WHERE fname LIKE 'j%' AND lname
> LIKE 'smi%'. Rreturns 55,874 rows in 7 seconds in QA.
> I am also concerned about time to populate FT index every day.
> Machine is dual Xeon 3.2 with 4GB ram, OS on RAID1 & SQL data on RAID5.
> Win
> 2003 Server and SQL2000, with SQL2005 later this year.
> Thanks much!
|||Thanks Hilary, but do you think a Full Text search would be faster searching
on single words?
"Hilary Cotter" wrote:

> Wow! That's fantastic speed for a like. I'd try change tracking. IIRC on a
> similar machine I was able to hit 2000 rows per minute with SQL 2000.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Bil Click" <BilClick@.discussions.microsoft.com> wrote in message
> news:FFF888F6-8310-4B1F-83B0-399830C17F53@.microsoft.com...
>
>
|||It will be for larger tables. I am not sure where the cut off it - i.e.
20,000 rows or 2 million.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Bil Click" <BilClick@.discussions.microsoft.com> wrote in message
news:FDFBA954-00ED-4444-A7EF-2B5547510343@.microsoft.com...[vbcol=seagreen]
> Thanks Hilary, but do you think a Full Text search would be faster
> searching
> on single words?
> "Hilary Cotter" wrote:

Sunday, March 11, 2012

Advanced Execution Plan Question

I have a scenario where I have 3 tables joined together. The first is the
"main" table that will define the result set. The second is a lookup table;
the main table has a foreign key to it. The third is a detail table; it has
a foreign key to the main table. Both the second and third tables are on
the right side on a LEFT OUTER JOIN. In this particular query, I only need
the detail data for a particular row if the lookup row exists. The reason
for this is that if the lookup row exists, it will be used to further filter
the result set based upon a value in the detail row. But if the lookup row
doesn't exist, there's no need to lookup the detail row. In order to
optimize the query I attempted to add an additional condition to the ON
clause joining the main and detail table saying only join if the lookup key
is not null:
SELECT
...
FROM
tbFilterTest F
LEFT JOIN tbFilterTestLookup L ON F.LookupID = L.LookupID
LEFT JOIN tbFilterTestDetail D ON L.LookupID IS NOT NULL
AND
F.FilterID
= D.FilterID
My hope was that in the join in the execution plan that there would be an
argument "L.LookupID <> null". What I'm ending up with in the actual
execution plan is the "L.LookupID <> null" is not in the join but its
showing up as an argument in the index s for the inner input, meaning
(I'm thinking) that the s is still getting executed and then the
"L.LookupID <> null" is being applied afterward, which means the s was
wasted if "L.LookupID = null".
Now I put together the 3 tables listed above to test this more closely.
Interestingly enough, the "L.LookupID <> null" argument WAS in the join and
not in the inner input, meaning the index s was not being performed on
the detail table when the lookup was null. HOWEVER, I noticed the inner
input (detail table) was having an index scan because I didn't have an index
on FilterID. Once I put the index on FilterID on the detail table, the
"L.LookupID <> null" moved back to the inner input (detail) index s!
I'm trying to understand the logic of this and also wonder if there is a way
to make the "L.LookupID <> null" always occur in the join. The actual
query I'm doing this for is very vital and regularly executed.
Below is DDL, sample data, and the query. Also is the index that "makes the
difference". If you execute the query without the index and then with the
index, you should see the "L.LookupID <> null" move from the join to the
inner input index s. Note that the sample query simply demonstrates "the
moving WHERE clause" in the execution plan. It does not demonstrate the
condition in my real query that moved me to try and accomplish this.
Thanks,
Mike
----
SET NOCOUNT ON
GO
IF EXISTS (SELECT * FROM sysobjects where name = 'tbFilterTestDetail')
DROP TABLE tbFilterTestDetail
GO
IF EXISTS (SELECT * FROM sysobjects where name = 'tbFilterTest')
DROP TABLE tbFilterTest
GO
IF EXISTS (SELECT * FROM sysobjects where name = 'tbFilterTestLookup')
DROP TABLE tbFilterTestLookup
GO
CREATE TABLE tbFilterTestLookup
(
LookupID int NOT NULL CONSTRAINT PK_FilterTestLookup PRIMARY KEY
)
GO
INSERT tbFilterTestLookup (LookupID) VALUES (1000)
INSERT tbFilterTestLookup (LookupID) VALUES (2000)
INSERT tbFilterTestLookup (LookupID) VALUES (3000)
INSERT tbFilterTestLookup (LookupID) VALUES (4000)
INSERT tbFilterTestLookup (LookupID) VALUES (5000)
INSERT tbFilterTestLookup (LookupID) VALUES (5001)
INSERT tbFilterTestLookup (LookupID) VALUES (5002)
INSERT tbFilterTestLookup (LookupID) VALUES (5003)
INSERT tbFilterTestLookup (LookupID) VALUES (5004)
INSERT tbFilterTestLookup (LookupID) VALUES (5005)
INSERT tbFilterTestLookup (LookupID) VALUES (5006)
INSERT tbFilterTestLookup (LookupID) VALUES (5007)
INSERT tbFilterTestLookup (LookupID) VALUES (5008)
INSERT tbFilterTestLookup (LookupID) VALUES (5009)
INSERT tbFilterTestLookup (LookupID) VALUES (5010)
INSERT tbFilterTestLookup (LookupID) VALUES (5011)
INSERT tbFilterTestLookup (LookupID) VALUES (5012)
INSERT tbFilterTestLookup (LookupID) VALUES (5013)
INSERT tbFilterTestLookup (LookupID) VALUES (5014)
GO
CREATE TABLE tbFilterTest
(
FilterID int NOT NULL CONSTRAINT PK_FilterTest PRIMARY KEY,
LookupID int NOT NULL
)
GO
INSERT tbFilterTest (FilterID, LookupID) VALUES (100, 10) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (200, 11) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (300, 12) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (400, 1000) -- Match
INSERT tbFilterTest (FilterID, LookupID) VALUES (500, 13) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (600, 14) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (700, 20) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (800, 20) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (900, 20) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (620, 20) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (730, 20) -- No match
INSERT tbFilterTest (FilterID, LookupID) VALUES (840, 20) -- No match
GO
CREATE TABLE tbFilterTestDetail
(
DetailID int IDENTITY (10000, 1) NOT NULL CONSTRAINT PK_tbFilterTestDetail
PRIMARY KEY,
FilterID int NOT NULL CONSTRAINT FK_tbFilterTestDetail_tbFilterTest
REFERENCES tbFilterTest (FilterID),
Descrip varchar(128) NOT NULL
)
GO
CREATE INDEX IX_tbFilterTestDetail_FilterID ON tbFilterTestDetail (FilterID)
GO
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (100, '100-1-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (100, '100-2-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (200, '200-1-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (200, '200-2-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (300, '200-1-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (300, '200-2-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (400, '400-1-match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (400, '400-2-match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (500, '500-1-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (500, '500-2-no match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (600, '600-1-match')
INSERT tbFilterTestDetail (FilterID, Descrip) VALUES (600, '600-2-match')
GO
-- Sample Query
SELECT
F.FilterID,
L.LookupID,
D.Descrip
FROM
tbFilterTest F
LEFT JOIN tbFilterTestLookup L ON F.LookupID = L.LookupID
LEFT JOIN tbFilterTestDetail D ON L.LookupID IS NOT NULL -- this is
the condition that moves in the execution plan
AND
F.FilterID =
D.FilterID
-- Magic index: try sample query w/ and w/o
CREATE INDEX IX_tbFilterTestDetail_FilterID ON tbFilterTestDetail (FilterID)
GOOn Thu, 18 Aug 2005 13:53:36 -0400, Mike Jansen wrote:
(snip)
>Both the second and third tables are on
>the right side on a LEFT OUTER JOIN. In this particular query, I only need
>the detail data for a particular row if the lookup row exists.
Hi Mike,
Then why use a LEFT OUTER JOIN' Outer joins should only be used if
you must retain rows from one table even though there is no match in the
other table. The standard join type (INNER JOIN) will retain only rows
that match (as per your request) and will probably perform faster as
well.
SELECT ...
FROM tbFilterTest AS F
INNER JOIN tbFilterTestLookup AS L
ON F.LookupID = L.LookupID
LEFT JOIN tbFilterTestDetail AS D
ON F.FilterID = D.FilterID
If you are sure that there will always be at least one matching row in
the FilterTestDetail table, change the second join to an INNER JOIN as
well. It won't change the results, but it might speed up the
performance.
Best, Hugo
--
(Remove _NO_ and _SPAM_ to get my e-mail address)