The below article is the intellectual property of Ashish Kohli. This is one such article which actually powers the ability of SQL. Give it a read guys.
Yes, you read that one right! One of the most fundamental machine learning algorithms out there is Linear Regression. In simple words, it is a technique to describe a relationship between a response(a.k.a dependent) variable and one or more explanatory (a.k.a independent) variable(s). After doing some reading into the “math” behind these algorithms, I realized that this can be easily achieved in SQL.
I don’t intend to turn this post into another “Machine Learning 101”. There are plenty of such articles out there that explain what Linear Regression is in a much better way, including some nitty-gritty details like it’s back-end optimization algorithm, i.e. Gradient Descent. I will thus try to keep this article as less “ML”ly as possible. However, I’d recommend this 9 byte-sized (typo intended!) article series that explains all this and some more in a really easy language. Also, for most of this post, I will be referring to the formulas and notations used in the hyperlinked article.
Alright! At this point, I hope that you already know about the concepts of Linear Regression and how Gradient Descent works. And thus you’ll also know that the relationship between response and explanatory variable(s) is explained by the following equation :
The goal of Linear Regression is to find the optimal value of θ (theta) that best describes the relationship between two variables and Gradient Descent is the way to do that. The equation given below summarizes how Gradient Descent arrives at the optimal value of θ(s).
Let’s list down the activities that are needed to be performed to arrive at optimal values of θ:
Start with random values of θ and calculate the value of hypothesis function (Hθ)
Fill in the values of θ and Hθ in the convergence equation to get new values of θ
Keep repeating Step 2 until the values of θ don’t change anymore
These values of θ correspond to the minimum cost (or error) for the Hypothesis function
Fill in the final values of θ in the hypothesis function to get Predicted values of the response variable
Step 1: Declaring & Initializing variables
We need 6 variables, each for a different purpose:
theta0, theta1 to store the current value of θ0, θ1
theta0_t, theta1_t to store temporary values of θ0 & θ1 before updating the original ones
var (short for variability) to check if the updated value of θ is approaching “near” the current value or not
alpha to store learning rate (read this answer at Quora to understand all about learning rate)
DECLARE @theta0 int; DECLARE @theta1 int; DECLARE @theta0_t int; DECLARE @theta1_t int; DECLARE @var DECIMAL(4,3); DECLARE @alpha DECIMAL(4,2); --Initial values SET @theta0 = 0; SET @theta1 = 1; SET @theta0_t = 1; SET @theta1_t = 0; SET @alpha = 0.1; SET @var = 0.01;
Step 2: Calculating values of Hθ and updated values of θ
--Calculating theta0 SELECT @theta0_t = @theta0 - (SUM(Outp)/(SELECT COUNT(*) FROM base))*@alpha FROM ( SELECT (@theta1*X + @theta0) - Y as Outp FROM base );--Calculating theta1 SELECT @theta1_t = @theta1 - (SUM(Outp)/(SELECT COUNT(*) FROM base))*@alpha FROM ( SELECT ((@theta1*X + @theta0) - Y)*X as Outp FROM base );
Step 3: Comparing if the updated values of θ are close to original θ or not
If the above condition is true, then we stop the process and finalize the values of θ. Otherwise, we keep repeating steps 2 & 3. Thus steps 2 & 3 need to be put inside a loop that runs as long as the updated and current values of θ are different.
WHILE (@theta0_t NOT BETWEEN @theta0-@var AND @theta0+@var) AND (@theta1_t NOT BETWEEN @theta1-@var AND @theta1+@var) BEGIN –Calculating theta0 SELECT @theta0_t = @theta0 – (SUM(Outp)/(SELECT COUNT(*) FROM base))*@alpha FROM ( SELECT (@theta1*X + @theta0) – Y as Outp FROM base );
--Calculating theta1 SELECT @theta1_t = @theta1 - (SUM(Outp)/(SELECT COUNT(*) FROM base))*@alpha FROM ( SELECT ((@theta1*X + @theta0) - Y)*X as Outp FROM base );
--Comparing thetas IF (@theta0_t BETWEEN @theta0-@var AND @theta0+@var) AND (@theta1_t BETWEEN @theta1-@var AND @theta1+@var) BEGIN SELECT @theta0 = @theta0_t; SELECT @theta1 = @theta1_t; BREAK; END ELSE BEGIN SELECT @theta0 = @theta0_t; SELECT @theta1 = @theta1_t; END END
The above loop will arrive at optimal values for θ. This is Gradient Descent in all it’s glory!
Step 4: Fill in the final values of θ in the hypothesis function to calculate predictions for the response variable
SELECT X,Y,@theta0+@theta1*X AS H_theta FROM base
And that’s it! We’ve built a machine learning algorithm in SQL with just a few lines of code!
Practical applications & final thoughts
Despite the onset of technological advancements in the field of Data Science, more often than not, every Data Scientist ends up working with legacy systems. In such cases, if the size of the data is huge, it becomes impractical to fetch it out of a legacy system (like SQL Server) into another environment for data science purposes.
Although I initially began this project as a weekend DIY, I feel this has bigger implications. This can be polished and packaged much better to improve its usability. Things like splitting of data into test & train, turning this into multi-variate linear regression will make this project much more practical. I would also love to hear thoughts of all of you on what can be improved.
Company – Affine Analytics Location – Bangalore Position – Senior Business Analyst Experience – 3+ years
Compensation – Best in the industry
Affine Analytics Interview Questions
Number of Rounds – 4
I received a call from the Technical HR who scheduled the telephonic round for the next day
Round1 – Telephonic Round (Mostly SQL and Project) I was asked to introduce myself and then the discussion went towards my recent project at Mu Sigma. We had a good discussion on Regression Techniques, a bit on statistics.
The project description was followed by few questions on SQL (the answers to these questions are present in various articles on the website, links are at the end of the interview)
1. What is the order of SQL query execution? 2. You have two tables with one column each. The table A has 5 values and all the values are 1 i.e. 1,1,1,1,1 and Table B has 3 values and all the values are 1 i.e. 1,1,1.
The telephonic round went for around 1 hour:- Introduction – 10 minutes Project – 30 minutes Questions – 20 minutes
I was shortlisted for the further rounds. All together the face-to-face interviews were divided into 3 rounds Round 1 – SQL and R/Python Round 2 – Statistics Round 3 – Case Study and HR questions
Business Case Study – There is a mobile company which is very popular in Other Asian countries. The company is planning to open it’s branch in the most popular mall of Bangalore. What should be the strategy of the company? How can you use freely available data to plan the marketing of the campaigns? How can you use Digital marketing to create campaigns for the company?
You can get your hand on our ebooks, we also have a 10 e-book bundle offer at Rs.549 where you get a total of 1400 questions. Comment below or mail at contact@thedatamonk.com for more information
When we want to execute a sequence of statements, we can
give it a name. Let’s define a function to take two numbers and return the
greater number.
>>> def greater(a,b): return a is a>b else b
5.Explain
Python List Comprehension.
The list comprehension in python is a way to
declare a list in one line of code. Let’s take a look at one such example.
>>> [i for i in range(1,11,2)] [1, 3, 5, 7, 9] >>> [i*2 for i in range(1,11,2)] [2, 6, 10, 14, 18]
6.How do you get all values from a Python dictionary?
We saw previously, to get all keys from a dictionary, we make a call to the keys() method. Similarly, for values, we use the method values().
>>> 'd' in {'a':1,'b':2,'c':3,'d':4}.values()
False
>>> 4 in {'a':1,'b':2,'c':3,'d':4}.values()
True
7. What is the difference between remove()
function and del statement?
You can use the
remove() function to delete a specific object in the list.
If you want to
delete an object at a specific location (index) in the list, you can either
use del or pop.
Note: You don’t
need to import any extra module to use these functions for removing an element
from the list.
We cannot use
these methods with a tuple because the tuple is different from the list.
8. How to remove leading whitespaces from a string in
the Python?
To remove
leading characters from a string, we can use lstrip() function. It is Python
string function which takes an optional char type parameter. If a parameter is
provided, it removes the character. Otherwise, it removes all the leading
spaces from the string.
string = " javatpoint " string2 = " javatpoint " print(string) print(string2) print("After stripping all leading whitespaces:") print(string.lstrip()) print(string2.lstrip())
9.Why do we use join() function in Python?
A. The join()
is defined as a string method which returns a string value. It is concatenated
with the elements of an iterable. It provides a flexible way to concatenate the
strings. See an example below.
str = "Rohan" str2 = "ab" # Calling function str2 = str.join(str2) # Displaying result print(str2) Output: aRohanb
10. What are the rules for a local and global variable in
Python?
A. In Python, variables that are only referenced inside a function are called implicitly global. If a variable is assigned a new value anywhere within the function’s body, it’s assumed to be a local. If a variable is ever assigned a new value inside the function, the variable is implicitly local, and we need to declare it as ‘global’ explicitly. To make a variable globally, we need to declare it by using global keyword. Local variables are accessible within local body only. Global variables are accessible anywhere in the program, and any function can access and modify its value.
1.How would you convert a string into an int in Python?
If a string contains only numerical characters, you can convert it
into an integer using the int() function.
>>> int(‘227’) 227
Let’s check the types: 1. >>>
type(‘227’)
<class ‘str’> 1. >>>
type(int(‘227’))
<class ‘int’>
2.What is difference between unique and
distinct?(90% asked Advanced SQL Interview Questions )
There
is no difference between unique and distinct keywords apart from one
difference.unique is applied before insertion and retrival.It consists of non
duplicate values.if unique constraint is given it does not take duplicate values.distinct
is used in retrieval it gives the suppressed row(ex if two rows are same it
will show single row and non duplicate row) therefore distinct is the
combination of suppressed duplicate and non duplicate rows.Specify DISTINCT or
UNIQUE if you want Oracle to return only one copy of each set of duplicate rows
selected (these two keywords are synonymous). Duplicate rows are those with
matching values for each expression in the select list.
3.What will be the output of following Query?
Query : select
case when null=null then ‘Amit’ Else ‘Pradnya’ from Table_Name;
In SQL null value is not equal to itself.So null=null is
false and the output of above query is ‘Pradnya’.
4.Which are
different Set operators in SQL?(100% asked Advanced SQL Interview Questions )
Set operators are nothing but the operators which are used
to connect two tables and fetch the records from the two tables.We need to
follow one condition that the table set 1 columns and table set 2 columns are
same and its datatype must be same.SQL Set Operators combines the result of 2
queries or components on to the single result.
Following are Set Operators in SQL:
1. Union
2.
Unionall
3. Intersect
4. Minus
5. How to select first 5 characters from First name in Employee table?
Oracle Query: Select
Substr(First_name,0,5) from Employee;
MS SQL: Select
Substr(First_name,1,5) from Employee;
MySQL: Select Substr(First_name,1,5)
from Employee;
6. What will be
the output of following query? Query :Select * from
(select ‘a’ union all select ‘b’) Q;
It will throw error because no values are selected in
Subquery.
7. Explain
co-related sub-query with example.
Fetch the Employees who have not assigned a single
department.
Select * from Employee E where Not exist
(Select Department_no From Department D where
E.Employee_id=D.Employee_ID);
Execution of query:
Step 1:
Select * from Employee E ;
It will fetch the all employees
Step 2:
The First Record of the Employee second query is executed
and output is given to first query.
(Select Department_no From Department D where
E.Employee_id=D.Employee_ID);
Step 3: Step 2 is
repeated until and unless all output is been fetched.
8. What is
difference between NVL,NVL2 and Nullif?
1.NVL :
NVL function substitutes a value when a null value is
encountered.
2.NVL2 :
NVL2 substitutes a value when a null value is encountered as
well as when a non-null value is encountered.
3.NULLIF:
NULLIF function compares expr1 and expr2. If expr1 and expr2
are equal, the NULLIF function returns NULL. Otherwise, it returns expr1.
9.What is
Index?What is use of index in SQL?
Index is optional structure associated with the table which
may or may not improve the performance of Query.In simple words suppose we want
to search the topic in to book we go to index page of that book and search the
topic which we want.Just like that to search the values from the table when
indexing is there you need not use the full table scan.
Indexes are used to improve the performance of the query.
10.What is the
difference between Having and Where clause?
Where clause is used to fetch data from a database that
specifies particular criteria whereas a Having clause is used along with ‘GROUP
BY’ to fetch data that meets particular criteria specified by the Aggregate
functions. Where clause cannot be used with Aggregate functions, but the Having
clause can.
Tuples – A tuple, like a list, is an ordered collection of values.
The difference. However, is that a tuple is immutable. This means that we
cannot change a value in it.
>>>
name=(‘Ayushi’,’Sharma’)
>>>
name[0]=’Avery’
Traceback (most recent call last):
File “<pyshell#129>”, line 1, in <module>
name[0]=’Avery’
TypeError: ‘tuple’ object does not support item assignment
10. What are Disctionary?
Dictionary – A dictionary is a data structure that holds key-value pairs. We
declare it using curly braces.
This is something which has been on my mind since a long time. We will be picking 10 questions per day and would like to simplify it. We will make sure that the complete article is covered in 10 minutes by the reader. There will be 100 posts in the coming 3 months.
The articles/questions will revolve around SQL, Statistics, Python/R, MS Excel, Statistical Modelling, and case studies.
The questions will be a mix of these topics to help you prepare for interviews
You can also contribute by framing 10 questions and sending it to contact@thedatamonk.com or messaging me on Linkedin.
The questions will be updated late in the night ~1-2 a.m. and will be posted on Linkedin as well.
Let’s see how many can we solve in the next 100 posts
1. How to find the minimum salary using subquery? -SELECT * FROM employee WHERE salary = (select MIN(salary) from employee);
2. How to find the second minimum salary? – SELECT * FROM employee WHERE salary = (SELECT MIN(salary) FROM employee > SELECT MIN(salary) FROM employee)
Similarly, find the third minimum salary
– SELECT * FROM employee WHERE salary = (SELECT MIN(salary) FROM employee > SELECT MIN(salary) FROM employee > SELECT MIN(salary) FROM employee)
3. The above query is too lengthy, write a query to get the third minimum salary with some other method.
– SELECT DISTINCT (salary) FROM emp e1 where 3 = (SELECT COUNT(DISTINCT salary) FROM emp e2 WHERE e1.sal >= e2.sal);
4. How to get 3 Min salaries? -SELECT DISTINCT salary FROM emp a WHERE 3 >= (SELECT COUNT(DISTINCT salary) FROM emp b WHERE a.salary >= b.salary);
6. Write a generic method to fetch the nth highest salary without TOP or Limit – SELECT Salary FROM Worker W1 WHERE n-1 = ( SELECT COUNT( DISTINCT ( W2.Salary ) ) FROM Worker W2 WHERE W2.Salary >= W1.Salary );
7. LAG(): Provides access to a row at a given physical offset that comes before the current row. Use this function in a SELECT statement to compare values in the current row with values in a previous row as specified by offset. Default offset is 1 if not specified. If Partition By clause is specified then it returns the offset Value in each partition after ordering the partition by Order By Clause.
Basically, lag() is used to create one more column in the table where you can get the previous value of the specified column
Col1
Col2
Lag_Col
a
10
Null
b
20
10
c
30
20
d
40
30
8.One more example
employee_number
last_name
first_name
salary
dept_id
12009
Sutherland
Barbara
54000
45
34974
Yates
Fred
80000
45
34987
Erickson
Neil
42000
45
45001
Parker
Sally
57500
30
75623
Gates
Steve
65000
30
SELECT dept_id, last_name, salary,
LAG (salary,1) OVER (ORDER BY salary) AS lower_salary
FROM employees;
dept_id
last_name
salary
lower_salary
45
Erickson
42000
NULL
45
Sutherland
54000
42000
30
Parker
57500
54000
30
Gates
65000
57500
45
Yates
80000
65000
9. LEAD() – Provides access to a row at a given physical offset that comes after the current row. Use this function in a SELECT statement to compare values in the current row with values in a subsequent row as specified by offset. Default offset is 1 if not specified. If Partition By clause is specified then it returns the offset Value in each partition after ordering the partition by Order By Clause
10. Which operator is used for Pattern Matching?
LIKE operator is used for pattern matching. It supports below wildcards. % : Matches any string of zero or more characters. _ : Matches any single character. [] : Matches any single character within the specified range ([a-f]) or set ([abcdef]). [^] : Matches any single character not within the specified range ([^a-f]) or set ([^abcdef])
This was the second set of 10 questions, if you want to learn more about the type of questions asked in different Data Science interviews then do try the below book:-
This is something which has been on my mind since a long time. We will be picking 10 questions per day and would like to simplify it. We will make sure that the complete article is covered in 10 minutes by the reader. There will be 100 posts in the coming 3 months.
The articles/questions will revolve around SQL, Statistics, Python/R, MS Excel, Statistical Modelling, and case studies.
The questions will be a mix of these topics to help you prepare for interviews
You can also contribute by framing 10 questions and sending it to contact@thedatamonk.com or messaging me on Linkedin.
The questions will be updated late in the night ~1-2 a.m. and will be posted on Linkedin as well.
Let’s see how many can we solve in the next 100 posts
1.Write the syntax to create a new column using Row Number over the Salary column
SELECT *, ROW_NUMBER() OVER (Order By Salary) as Row_Num FROM Employee
Output
Emp. ID
Name
Salary
Row_Num
232
Rakshit
30000
1
543
Rahul
30000
2
124
Aman
40000
3
123
Amit
50000
4
453
Sumit
50000
5
2. What is PARTITION BY clause? PARTITION BY clause is used to create a partition of ranking in a table. If you partition by Salary in the above table, then it will provide a ranking based on each unique salary. Example below:-
SELECT *, ROW_NUMBER() OVER (PARTITION BY Salary ORDER BY Salary) as Row_Num
Emp. ID
Name
Salary
Row_Num
232
Rakshit
30000
1
543
Rahul
30000
2
124
Aman
40000
1
123
Amit
50000
1
453
Sumit
50000
2
3. What is a RANK() function? How is it different from ROW_NUMBER()? – RANK() function gives ranking to a row based on the value on which you want to base your ranking. If there are equal values, then the rank will be repeated and the row following the repeated values will skip as many ranks as there are repeated values row. Confused?? Try out the example below:-
SELECT *, RANK() OVER (ORDER BY Salary) as Row_Num FROM Employee
Output
Emp. ID
Name
Salary
Row_Num
232
Rakshit
30000
1
543
Rahul
30000
1
124
Aman
40000
3
123
Amit
50000
4
453
Sumit
50000
4
As you can see, the rank 2 has been skipped because there were two employees with the same Salary and the result is ordered in ascending order by default.
4. What is Dense Ranking? – DENSE_RANK() is similar to the RANK() function but it does not skip any rank, so if there are two equal values then both will be termed as 1, the third value will be termed as 3 and not 2.
Syntax:- SELECT *, DENSE_RANK() OVER (PARTITION BY Salary ORDER BY Salary) as Row_Num FROM Employee
Output:-
Emp. ID
Name
Salary
Row_Num
232
Rakshit
30000
1
543
Rahul
30000
1
124
Aman
40000
3
123
Amit
50000
4
453
Sumit
50000
4
432
Nihar
60000
6
5.What is NTILE() function? -NTILE() is similar to percentile NTILE(3) will divide the data in 3 parts.
SELECT *, NTILE() OVER (ORDER BY Salary) as Ntile FROM Employee
The number of rows should be 6/3 = 2, therefore we need to divide the 2 rows for each percentile
Emp. ID
Name
Salary
Ntile
232
Rakshit
30000
1
543
Rahul
30000
1
124
Aman
40000
2
123
Amit
50000
2
453
Sumit
50000
3
432
Nihar
60000
3
6. How to get the second highest salary from a table? Select MAX(Salary) from Employee Where Salary NOT IN (SELECT MAX(Salary) from Employee)
7. Find the 3rd Maximum salary in the employee table -Select distinct sal from emp e1 where 3 = ((select count(distinct sal) from emp e2 where e1.sal <= e2.sal);
8. Get all employee detail from EmployeeDetail table whose “FirstName” not start with any single character between ‘a-p’ – SELECT * FROM EmployeeDetail WHERE FirstName like ‘[^a-p]%’
9. How to fetch only even rows from a table? -The best way to do it is by adding a row number using ROW_NUMBER() and then pulling the alternate row number using row_num%2 = 0
Suppose, there are 3 columns in a table i.e. student_ID, student_Name, student_Grade. Pull the even rows
SELECT * FROM ( SELECT *, ROW_NUMBER() OVER (ORDER BY student_ID) as row_num FROM student) x WHERE x.row_num%2=0
10. How to fetch only odd rows from the same table? -Simply apply the x.row_num%2 <> 0 to get the odd rows
SELECT * FROM ( SELECT *, ROW_NUMBER() OVER (ORDER BY student_ID) as row_num FROM student) x WHERE x.row_num%2 <> 0
Let us know if you think I need to change any answer here.
Q1.
What are the parts of Microsoft self-service business intelligence solution?
Microsoft
has two parts for Self-Service BI
Excel
BI
Toolkit
It
Allows users to create an interactive report by importing data from different
sources and model data according to report requirement.
Power
BI
It
is The online solution that enables you to share the interactive reports and
queries that you have created using the Excel BI Toolkit.
Q2.
What is self-service business intelligence?
Self-Service Business Intelligence
(SSBI)
SSBI
is an approach to data analytics that enables business users to filter,
segment, and, analyze their data, without the in-depth technical knowledge in
statistical analysis, business intelligence (BI).
SSBI
has made it easier for end users to access their data and create various
visuals to get better business insights.
Anybody
who has a basic understanding of the data can create reports to build intuitive
and shareable dashboards.
Power BI is a cloud-based data sharing
environment. Once you have developed reports using Power Query, Power Pivot and
Power View, you can share your insights with your colleagues. This is where
Power BI enters the equation. Power BI, which technically is an aspect of
SharePoint online, lets you load Excel workbooks into the cloud and share them
with a chosen group of co-workers. Not only that, but your colleagues can
interact with your reports to apply filters and slicers to highlight data. They
are completed by Power BI, a simple way of sharing your analysis and insights
from the Microsoft cloud.
Power BI features allow you to:
Share
presentations and queries with your colleagues.
Update
your Excel file from data sources that can be on-site or in the cloud.
Display
the output on multiple devices. This includes PCs, tablets, and HTML 5-enabled
mobile devices that use the Power BI app.
Query
your data using natural language processing (or Q&A, as it is known).
Q4. What is Power BI Desktop?
Power BI Desktop is a free desktop
application that can be installed right on your own computer. Power BI Desktop
works cohesively with the Power BI service by providing advanced data
exploration, shaping, modeling, and creating report with highly interactive
visualizations. You can save your work to a file or publish your data and
reports right to your Power BI site to share with others.
Q5. What data sources can Power BI connect to?
The list of data sources for Power BI
is extensive, but it can be grouped into the following:
Files:
Data can be imported from Excel (.xlsx, xlxm), Power BI Desktop files (.pbix)
and Comma Separated Value (.csv).
Content
Packs: It is a collection of related documents or files that are stored as a
group. In Power BI, there are two types of content packs, firstly those from
services providers like Google Analytics, Marketo or Salesforce and secondly
those created and shared by other users in your organization.
Connectors
to databases and other datasets such as Azure SQL, Database and SQL, Server
Analysis Services tabular data, etc.
Q6. What are Building Blocks in Power BI?
The following are the Building Blocks
(or) key components of Power BI:
Visualizations: Visualization is a visual
representation of data.
Example: Pie Chart, Line Graph, Side by Side Bar Charts, Graphical Presentation
of the source data on top of Geographical Map, Tree Map, etc.
Datasets: Dataset is a collection of data that
Power BI uses to create its visualizations.
Example: Excel sheets, Oracle or SQL server tables.
Reports: Report is a collection of
visualizations that appear together on one or more pages.
Example: Sales by Country, State, City Report, Logistic Performance report,
Profit by Products report etc.
Dashboards: Dashboard is single layer presentation
of multiple visualizations, i.e we can integrate one or more visualizations
into one page layer.
Example: Sales dashboard can have pie charts, geographical maps and bar charts.
Tiles: Tile is a single visualization in a
report or on a dashboard.
Example: Pie Chart in Dashboard or Report.
Power
BI provides variety of option to filter report, data and visualization. The
following are the list of Filter types.
Visual-level Filters: These filters work on only an
individual visualization, reducing the amount of data that the visualization
can see. Moreover, visual-level filters can filter both data and calculations.
Page-level Filters: These filters work at the report-page
level. Different pages in the same report can have different page-level
filters.
Report-level Filters: There filters work on the entire
report, filtering all pages and visualizations included in the report.
We know that Power BI visual have
interactions feature, which makes filtering a report a breeze. Visual
interactions are useful, but they come with some limitations:
The
filter is not saved as part of the report. Whenever you open a report, you can
begin to play with visual filters but there is no way to store the filter in
the saved report.
The
filter is always visible. Sometimes you want a filter for the entire report,
but you do not want any visual indication of the filter being applied.
Q8. What are content packs in Power BI?
Content packs for services are
pre-built solutions for popular services as part of the Power BI experience. A
subscriber to a supported service, can quickly connect to their account from
Power BI to see their data through live dashboards and interactive reports that
have been pre-built for them. Microsoft has released content packs for popular
services such as Salesforce.com, Marketo, Adobe Analytics, Azure Mobile
Engagement, CircuitID, comScore Digital Analytix, Quickbooks Online, SQL Sentry
and tyGraph. Organizational content packs provide users, BI professionals, and
system integrator the tools to build their own content packs to share
purpose-built dashboards, reports, and datasets within their organization.
Q9. What is DAX?
To do basic calculation and data
analysis on data in power pivot, we use Data Analysis Expression (DAX). It is
formula language used to compute calculated column and calculated field.
DAX
works on column values.
DAX
can not modify or insert data.
We
can create calculated column and measures with DAX but we can not calculate rows using DAX.
Sample DAX formula syntax:
For the measure named Total Sales,
calculate (=) the SUM of values in the [SalesAmount] column in the Sales table.
A- Measure Name
B- = – indicate beginning of formula
C- DAX Function
D- Parenthesis for Sum Function
E- Referenced Table
F- Referenced column name
Q9. What are some of the DAX functions?
Below are some of the most commonly
used DAX function:
The FILTER function returns a table
with a filter condition applied for each of its source table rows. The FILTER
function is rarely used in isolation, it’s generally used as a parameter to
other functions such as CALCULATE.
FILTER
is an iterator and thus can negatively impact performance over large source
tables.
Complex
filtering logic can be applied such as referencing a measure in a filter
expression.
FILTER(MyTable,[SalesMetric]
> 500)
Q12. What are the functions and limitations of DAX?
These are the only functions that allow
you modify filter context of measures or tables.
Add
to existing filter context of queries.
Override
filter context from queries.
Remove
existing filter context from queries.
Limitations:
Filter
parameters can only operate on a single column at a time.
Filter
parameters cannot reference a metric.
Q9. What is SUMMARIZE() and SUMMARIZECOLUMNS() DAX?
SUMMARIZE()
Main
group by function in SSAS.
Recommended
practice is to specify table and group by columns but not metrics.You can use
ADDCOLUMNS function.
SUMMARIZECOLUMNS
New
group by function for SSAS and Power BI Desktop; more efficient.
Specify
group by columns, table, and expressions.
Q14. What are some benefits of using Variables in DAX ?
Below are some of the benefits:
By
declaring and evaluating a variable, the variable can be reused multiple times
in a DAX expression, thus avoiding additional queries of the source database.
Variables
can make DAX expressions more intuitive/logical to interpret.
Variables
are only scoped to their measure or query, they cannot be shared among
measures, queries or be defined at the model level.
Q15. How would you create trailing X month metrics via DAX against
a non-standard calendar?
The
solution will involve:
CALCULATE
function to control (take over) filter context of measures.
ALL
to remove existing filters on the date dimension.
FILTER
to identify which rows of the date dimension to use.
Alternatively, CONTAINS may be used:
CALCULATE(FILTER(ALL(‘DATE’),…….))
Q16. What are the different BI add-in to Excel ?
Below are the most important BI add-in
to Excel:
Power
Query: It helps in finding, editing and loading external data.
Power
Pivot: Its mainly used for data modeling and analysis.
Power
View: It is used to design visual and interactively reports.
Power
Map: It helps to display insights on 3D Map.
Q17. What is Power Pivot?
Power Pivot is an add-in for Microsoft
Excel 2010 that enables you to import millions of rows of data from multiple
data sources into a single Excel workbook. It lets you create relationships
between heterogeneous data, create calculated columns and measures using
formulas, build PivotTables and PivotCharts. You can then further analyze the
data so that you can make timely business decisions without requiring IT
assistance.
Q18. What is Power Pivot Data Model?
It is a model that is made up of data
types, tables, columns, and table relations. These data tables are typically
constructed for holding data for a business entity.
Q19. What is xVelocity in-memory analytics engine used in Power Pivot?
The main engine behind power pivot is
the xVelocity in-memory analytics engine. It can handle large amount of data
because it stores data in columnar databases, and in memory analytics which
results in faster processing of data as it loads all data to RAM memory.
Q20. What are some of differences in data modeling between Power
BI Desktop and Power Pivot for Excel?
Here are some of the differences:
Power
BI Desktop supports bi-directional cross filtering relationships, security,
calculated tables, and Direct Query options.
Power
Pivot for Excel has single direction (one to many) relationships, calculated
columns only, and supports import mode only. Security roles cannot be defined
in Power Pivot for Excel.
Q21. Can we have more than one active relationship between two
tables in data model of power pivot?
No, we cannot have more than one active
relationship between two tables. However, can have more than one relationship
between two tables but there will be only one active relationship and many
inactive relationship. The dotted lines are inactive and continuous line are
active.
Q22. What is Power Query?
Power query is a ETL Tool used to
shape, clean and transform data using intuitive interfaces without having to
use coding. It helps the user to:
Import
Data from wide range of sources from files, databases, big data, social media
data, etc.
Join
and append data from multiple data sources.
Shape
data as per requirement by removing and adding data.
Q23. What are the data destinations for Power Queries?
There are two destinations for output
we get from power query:
Load
to a table in a worksheet.
Load
to the Excel Data Model.
Q24. What is query folding in Power Query?
Query folding is when steps defined in
Power Query/Query Editor are translated into SQL and executed by the source
database rather than the client machine. It’s important for processing
performance and scalability, given limited resources on the client machine.
Q25. What are some common Power Query/Editor Transforms?
Changing Data Types, Filtering Rows,
Choosing/Removing Columns, Grouping, Splitting a column into multiple columns,
Adding new Columns ,etc.
Q26. Can SQL and Power Query/Query Editor be used together?
Yes, a SQL statement can be defined as
the source of a Power Query/M function for additional processing/logic. This
would be a good practice to ensure that an efficient database query is passed
to the source and avoid unnecessary processing and complexity by the client
machine and M function.
Q28. What are query parameters and Power BI templates?
Query parameters can be used to provide
users of a local Power BI Desktop report with a prompt, to specify the values
they’re interested in.
The
parameter selection can then be used by the query and calculations.
PBIX
files can be exported as Templates (PBIT files).
Templates
contain everything in the PBIX except the data itself.
Parameters and templates can make it
possible to share/email smaller template files and limit the amount of data
loaded into the local PBIX files, improving processing time and experience.
Q29. Which language is used in Power Query?
A new programming language is used in
power query called M-Code. It is easy to use and similar to other languages.
M-code is case sensitive language.
Q30. Why do we need Power Query when Power Pivot can import data
from mostly used sources?
Power Query is a self-service ETL
(Extract, Transform, Load) tool which runs as an Excel add-in. It allows users
to pull data from various sources, manipulate said data into a form that suits
their needs and load it into Excel. It is most optimum to use Power Query over
Power Pivot as it lets you not only load the data but also manipulate it as per
the users needs while loading.
Q31. What is Power Map?
Power Map is an Excel add-in that
provides you with a powerful set of tools to help you visualize and gain
insight into large sets of data that have a geo-coded component. It can help
you produce 3D visualizations by plotting upto a million data points in the
form of column, heat, and bubble maps on top of a Bing map. If the data is time
stamped, it can also produce interactive views that display, how the data
changes over space and time.
Q32. What are the primary requirement for a table to be used in
Power Map?
For a data to be consumed in power map
there should be location data like:
Latitude/Longitude
pair
Street,
City, Country/Region, Zip Code/Postal Code, and State/Province, which can be
geolocated by Bing
The primary requirement for the table
is that it contains unique rows. It must also contain location data, which can
be in the form of a Latitude/Longitude pair, although this is not a requirement.
You can use address fields instead, such as Street, City, Country/Region, Zip
Code/Postal Code, and State/Province, which can be geolocated by Bing.
Q33. What are the data sources for Power Map?
The data can either be present in Excel
or could be present externally. To prepare your data, make sure all of the data
is in Excel table format, where each row represents a unique record. Your
column headings or row headings should contain text instead of actual data, so
that Power Map will interpret it correctly when it plots the geographic
coordinates. Using meaningful labels also makes value and category fields
available to you when you design your tour in the Power Map Tour Editor pane.
To use a table structure which more
accurately represents time and geography inside Power Map, include all of the
data in the table rows and use descriptive text labels in the column headings,
like this:
In case you wish to load your data from
an external source:
In
Excel, click Data > the connection you want in the Get External Data group.
Follow
the steps in the wizard that starts.
On
the last step of the wizard, make sure Add this data to the Data Model is
checked.
Q34. What is Power View?
Ans: Power View is a data visualization
technology that lets you create interactive charts, graphs, maps, and other
visuals which bring your data to life. Power View is available in Excel,
SharePoint, SQL Server, and Power BI.
The following pages provide details
about different visualizations available in Power View:
Charts
Line
charts
Pie
charts
Maps
Tiles
Cards
Images
Tables
Power
View
Multiples
Visualizations
Bubble
and scatter charts
Key
performance indicators (KPIs)
Q35. What is Power BI Designer?
Ans: It is a stand alone application
where we can make Power BI reports and then upload it to Powerbi.com, it does
not require Excel. Actually, it is a combination of Power Query, Power Pivot,
and Power View.
Q36. Can we refresh our Power BI reports once uploaded to cloud
(Share point or Powebi.com)?
Ans: Yes we can refresh our reports
through Data Management gateway(for sharepoint), and Power BI Personal
gateway(for Powerbi.com)
Q37. What are the different types of refreshing data for our
published reports?
Ans: There are four main types of
refresh in Power BI. Package refresh, model or data refresh, tile refresh and
visual container refresh.
Package refresh
This synchronizes your Power BI
Desktop, or Excel, file between the Power BI service and OneDrive, or
SharePoint Online. However, this does not pull data from the original data
source. The dataset in Power BI will only be updated with what is in the file
within OneDrive, or SharePoint Online.
Model/data refresh
It referrs to refreshing the dataset,
within the Power BI service, with data from the original data source. This is
done by either using scheduled refresh, or refresh now. This requires a gateway
for on-premises data sources.
Tile refresh
Tile refresh updates the cache for tile
visuals, on the dashboard, once data changes. This happens about every fifteen
minutes. You can also force a tile refresh by selecting the ellipsis (…) in the
upper right of a dashboard and selecting Refresh dashboard tiles.
Visual container refresh
Refreshing the visual container updates
the cached report visuals, within a report, once the data changes.
Q38. Is Power BI available on-premises?
No, Power BI is not available as a
private, internal cloud service. However, with Power BI and Power BI Desktop,
you can securely connect to your own on-premises data sources. With the On-premises
Data Gateway, you can connect live to your on-premises SQL Server Analysis
Services, and other data sources. You can also scheduled refresh with a
centralized gateway. If a gateway is not available, you can refresh data from
on-premises data sources using the Power BI Gateway – Personal.
Q39. What is data management gateway and Power BI personal
gateway?
Gateway acts a bridge between
on-premises data sources and Azure cloud services.
Personal Gateway:
Import
Only, Power BI Service Only, No central monitoring/managing.
Can
only be used by one person (personal); can’t allow others to use this gateway.
On-Premises Gateway:
Import
and Direct Query supported.
Multiple
users of the gateway for developing content.
Central
monitoring and control.
Q40. What is Power BI Q&A?
Power BI Q&A is a natural language
tool which helps in querying your data and get the results you need from it.
You do this by typing into a dialog box on your Dashboard, which the engine
instantaneously generates an answer similar to Power View. Q&A interprets
your questions and shows you a restated query of what it is looking from your
data. Q&A was developed by Server and Tools, Microsoft Research and the
Bing teams to give you a complete
feeling of truly exploring your data.
41). What are some ways that Excel
experience can be leveraged with Power BI?
Below are some of the ways through
which we can leverage Power BI:
The
Power BI Publisher for Excel:
Can
be used to pin Excel items (charts, ranges, pivot tables) to Power BI Service.
Can
be used to connect to datasets and reports stored in Power BI Service.
Excel
workbooks can be uploaded to Power BI and viewed in the browser like Excel
Services.
Excel
reports in the Power BI service can be shared via Content Packs like other
reports.
Excel
workbooks (model and tables) can be exported to service for PBI report
creation.
Excel
workbook Power Pivot models can be imported to Power BI Desktop models.
Q42. What is a calculated column in Power BI and why would you use
them?
Calculated Columns are DAX expressions
that are computed during the model’s processing/refresh process for each row of
the given column and can be used like any other column in the model.
Calculated columns are not compressed
and thus consume more memory and result in reduced query performance. They can
also reduce processing/refresh performance if applied on large fact tables and
can make a model more difficult to maintain/support given
that the calculated column is not
present in the source system.
Q43. How is data security implemented in Power BI ?
Power BI can apply Row Level Security
roles to models.
A
DAX expression is applied on a table filtering its rows at query time.
Dynamic
security involves the use of USERNAME functions in security role definitions.
Typically
a table is created in the model that relates users to specific dimensions and a
role.
Q44. What are many-to-many relationships and how can they be
addressed in Power BI ?
Many to Many relationships involve a
bridge or junction table reflecting the combinations of two dimensions (e.g.
doctors and patients). Either all possible combinations or those combinations
that have occurred.
Bi-Directional
Crossfiltering relationships can be used in PBIX.
CROSSFILTER
function can be used in Power Pivot for Excel.
DAX
can be used per metric to check and optionally modify the filter context.
Q45. Why might you have a table in the model without any
relationships to other tables?
There are mainly 2 reasons why we would
have tables without relations in our model:
A
disconnected table might be used to present the user with parameter values to
be exposed and selected in slicers (e.g. growth assumption.)
DAX
metrics could retrieve this selection and use it with other
calculations/metrics.
A
disconnected table may also be used as a placeholder for metrics in the user
interface.
It
may not contain any rows of data and its columns could be hidden but all
metrics are visible.
46). What is the Power BI Publisher for Excel?
You can use Power BI publisher for
Excel to pin ranges, pivot tables and charts to Power BI.
The
user can manage the tiles – refresh them, remove them, in Excel.
Pinned
items must be removed from the dashboard in the service (removing in Excel only
deletes the connection).
The
Power BI Publisher for Excel can also be used to connect from Excel to datasets
that are hosted in the Power BI Service.
An
Excel pivot table is generated with a connection (ODC file) to the data in
Azure.
Q47. What are the differences between a Power BI Dataset, a
Report, and a Dashboard?
Dataset: The source used to create
reports and visuals/tiles.
A
data model (local to PBIX or XLSX) or model in an Analysis Services Server
Data
could be inside of model (imported) or a Direct Query connection to a source.
Report: An individual Power BI Desktop
file (PBIX) containing one or more report pages.
Built
for deep, interactive analysis experience for a given dataset (filters,
formatting).
Each
Report is connected to atleast one dataset
Each
page containing one or more visuals or tiles.
Dashboard: a collection of visuals or
tiles from different reports and, optionally, a pinned.
Built
to aggregate primary visuals and metrics from multiple datasets.
Q48. What are the three Edit Interactions options of a visual tile
in Power BI Desktop?
The 3 edit interaction options are Filter, Highlight, and None.
Filter: It completely filter a
visual/tile based on the filter selection of another visual/tile.
Highlight: It highlight only the
related elements on the visual/tile, gray out the non-related items.
None: It ignore the filter selection
from another tile/visual.
Q49. What are some of the differences in report authoring
capabilities between using a live or direct query connection such as to an
Analysis Services model, relative to working with a data model local to the
Power BI Desktop file?
With a data model local to the PBIX
file (or Power Pivot workbook), the author has full control over the queries,
the modeling/relationships, the metadata and the metrics.
With a live connection to an Analysis
Services database (cube) the user cannot create new metrics, import new data,
change the formatting of the metrics, etc – the user can only use the
visualization, analytics, and formatting available on the report canvas.
With a direct query model in Power BI
to SQL Server, for example, the author has access to the same features (and
limitations) available to SSAS Direct
Query mode.
Only
one data source (one database on one server) may be used, certain DAX functions
are not optimized, and the user cannot use Query Editor functions that cannot
be translated into SQL statements.
Q50. How does SSRS integrate with Power BI?
Below are some of the way through which
SSRS can be integrated with Power BI:
Certain
SSRS Report items such as charts can be pinned to Power BI dashboards.
Clicking
the tile in Power BI dashboards will bring the user to the SSRS report.
A
subscription is created to keep the dashboard tile refreshed.
Power
BI reports will soon be able to be published to SSRS portal
The Data Monk services
We are well known for our interview books and have 70+ e-book across Amazon and The Data Monk e-shop page . Following are best-seller combo packs and services that we are providing as of now
YouTube channel covering all the interview-related important topics in SQL, Python, MS Excel, Machine Learning Algorithm, Statistics, and Direct Interview Questions Link –The Data Monk Youtube Channel
Website – ~2000 completed solved Interview questions in SQL, Python, ML, and Case Study Link –The Data Monk website
E-book shop – We have 70+ e-books available on our website and 3 bundles covering 2000+ solved interview questions. Do check it out Link –The Data E-shop Page
Instagram Page – It covers only Most asked Questions and concepts (100+ posts). We have 100+ most asked interview topics explained in simple terms Link –The Data Monk Instagram page
A. A data sample is a
set of data collected and the world selected from
a statistical population by a defined procedure. The elements of
a sample are known
as sample points, sampling units or observations.
Q2. Define Population.
A. In statistics, population refers
to the total set of observations that can be made. For example, if we are
studying the weight of adult women, the population is the set of
weights of all the women in the world
Q3. What is a Data Point?
A. In statistics, a data point (or
observation) is a set of one or more measurements on a single member of a
statistical population.
Q4. Explain Data Sets.
A. Data sets usually
come from actual observations obtained by sampling a statistical population,
and each row corresponds to the observations on one element of that
population. Data sets may further be generated by algorithms for the
purpose of testing certain kinds of software.
Q5. What is meant by the term Inferential Statistics?
A. Inferential statistics use a random sample of
data taken from a population to describe and make inferences about the
population. Inferential statistics are valuable when examination of each member
of an entire population is not convenient or possible.
Q6. Give an
example of Inferential Statistics
A. You asked five of your
classmates about their height. On the basis of this information, you stated
that the average height of all students in your university or college is 67
inches.
Q7. What is Descriptive Statistics?
A. Descriptive statistics are
brief descriptive coefficients that summarize a given data set, which can be
either a representation of the entire or a sample of a population. Descriptive
statistics are broken down into measures of central tendency and measures of
variability (spread).
Q8. What is the range of data?
A1. It tells us how much the data is spread across in a set.
In other words, it is defined as the difference between the highest and the
lowest value present in the set.
X=[2 3 4 4 3 7 9]
Range(x)%return (9-2)=7
Q9. Define Measurement.
A. Data can be classified as being on one of four scales:
nominal
ordinal
interval
ratio
Each level of measurement has some important properties that are useful
to know. For example, only the ratio scale has meaningful zeros.
Q10. What is a Nominal Scale?
A. Nominal variables (also called categorical variables) can be placed into
categories. They don’t have a numeric value and so cannot be added,
subtracted, divided or multiplied. They also have no order; if they appear to
have an order then you probably have ordinal variables instead
Q11. What is an Ordinal Scale?
A. The ordinal scale contains things that you can place in order. For
example, hottest to coldest, lightest to heaviest, richest to poorest.
Basically, if you can rank data by 1st, 2nd, 3rd place (and so on), then you
have data that’s on an ordinal scale.
Q12. What is an Interval Scale?
A. An interval scale has ordered numbers with
meaningful divisions. Temperature is on the interval scale: a difference of 10
degrees between 90 and 100 means the same as 10 degrees between 150 and 160.
Compare that to high school ranking (which is ordinal), where the difference
between 1st and 2nd might be .01 and between 10th and 11th .5. If you have
meaningful divisions, you have something on the interval scale.
Q13. Explain Ratio Scale.
A. The ratio scale is exactly the same as the
interval scale with one major difference: zero is meaningful. For example, a
height of zero is meaningful (it means you don’t exist). Compare that to a
temperature of zero, which while it exists, it doesn’t mean anything in
particular.
Q14. What do you mean by Bayesian?
A. Bayesians condition on the data observed and considered
the probability distribution on the hypothesis. Bayesian
statistics provides us with mathematical tools to rationally update our
subjective beliefs in light of new data or evidence.
Q15. What is Frequentist?
A. Frequentists condition on a hypothesis of choice and
consider the probability distribution on the data, whether observed or not.
Frequentist statistics uses rigid frameworks, the type of frameworks that you
learn in basic statistics, like:
A. In statistical significance testing, it is the
probability of obtaining a test statistic at least as extreme as the one that
was actually observed, assuming that the null hypothesis is true.
Q17. What is a Confidence Interval?
A. A confidence interval,
in statistics, refers to the probability that a population parameter will
fall between two set values for a certain proportion of times. Confidence
intervals measure the degree of uncertainty or certainty in a sampling
method.
Q18. Explain Hypothesis Testing.
A. Hypothesis testing is an act
in statistics whereby an analyst tests an
assumption regarding a population parameter. The methodology employed by the
analyst depends on the nature of the data used and the reason for the analysis.
Hypothesis testing is used to infer the result of a hypothesis performed on
sample data from a larger population.
Q19. What is likelihood?
A. The probability of some observed outcomes given a set of
parameter values is regarded as the likelihood of the set of parameter values
given the observed outcomes.
Q20. What is sampling?
A. Sampling is that part of statistical practice concerned
with the selection of an unbiased or random subset of individual observations
within a population of individuals intended to yield some knowledge about the
population of concern.
Q21. What are
Sampling Methods?
A. There are 4 sampling methods:
Simple Random
Systematic
Cluster
Stratified
Q22.What is Mode?
A. The mode of a data sample is the element that occurs the
most number of times in the data collection.
X=[1 2 4 4 4 4
5 5]
Mode(x)% return
3
Q23. What is Median?
A. It is describes as the numeric value that separates the
lower half of sample of a probability from the upper half. It can b easily
calculated by arranging all the samples from highest to lowest (or vice-versa)
and picking the middle one.
X=[2 4 1 3 4 4
3]
X=[1 2 3 3 4 4
4]
Median(x)%
return 3
Q24. What is meant by Quartile?
A. It is a type of quantile that divides the data points
into four or less equal parts(quarters). Each
quartile contains 25% of the total observations. Generally, the data is
arranged from smallest to largest.
Q25. What is Moment?
A. It is the quantitative measure of the shape of a set of
points. It comprises of a set of statistical parameters to measure a
distribution. Four moments are commonly used:
Mean
Skewness
Variance
Kurtosis
Q26. What is the Mean of data?
A. The statistical
mean refers to the mean or average that is used to derive the
central tendency of the data in question. It is determined by adding all the
data points in a population and then dividing the total by the number of
points.
X=[1 2 3 3 6]
Sum=1+2+3+3+6=15
Mean(x)%return (sum/5)=3
Q27. Define Skewness.
A. Skewness is a measure of the asymmetric of the data
around the sample mean. It it is negative, the data are spread out more to the
left side of the mean than to the right. The vice-versa also stands true.
Q28. What is Variance?
A. It describes how far the value lies from the Mean. A small variance indicates that the data points tend to be very close to
the mean, and to each other. A high variance indicates that the data points are
very spread out from the mean, and from one another. Variance is the average of the squared distances from
each point to the mean.
Q29. Define Standard Deviation.
A. In statistics, the standard deviation is a
measure of the amount of variation or dispersion of a set of values. A low
standard deviation indicates that the values tend to be close to the mean of
the set, while a high standard deviation indicates that the values are spread
out over a wider range.
Q30. What is Kurtosis?
A. Kurtosis is a measure of how outlier-prone a distribution
is. In other words, kurtosis identifies
whether the tails of a given distribution contain extreme values.
Q31. What is meant by Covariance?
A. Covariance measures the
directional relationship between the returns on two assets. A positive covariance means that asset returns move
together while a negative covariance means they move inversely. Covariance
is calculated by analyzing at-return surprises (standard deviations from the expected return) or by multiplying
the correlation between the two variables by the standard deviation of each variable.
It gives the measure of how much two variable change together.
Q32. What is Alternative Hypothesis?
A. The Alternative hypothesis
(denoted by H1 ) is the statement that must be true if the null hypothesis is
false.
Q33. Explain Significance Level.
A. The probability of rejecting
the null hypothesis when it is called the significance level α , and very
common choices are α = 0.05 and α = 0.01.
Q34. Do you know what is Binary search?
A. For binary search, the array
should be arranged in ascending or descending order. In each step, the
algorithm compares the search key value with the key value of the middle
element of the array. If the keys match, then a matching element has been found
and its index, or position, is returned. Otherwise, if the search key is less
than the middle element’s key, then the algorithm repeats its action on the
sub-array to the left of the middle element or, if the search key is greater,
on the sub-array to the right.
Q35. Explain Hash Table.
A. A hash table is a data structure used to implement an
associative array, a structure that can map keys to values. A hash table uses a
hash function to compute an index into an array of buckets or slots, from which
the correct value can be found.
Q36.
What is Null Hypothesis?
A. The null hypothesis
(denote by H0 ) is a statement about the value of a population parameter
(such as mean), and it must contain the condition of equality and must be
written with the symbol =, ≤, or ≤.
Q37.
When You Are
Creating A Statistical Model How Do You Prevent Over-fitting?
A. It can
be prevented by cross-validation
Q38. What do you mean by Cross-vlidation?
A. Cross-validation, it’s a
model validation techniques for assessing how the results of
a statistical analysis (model) will generalize to an independent data
set. It is mainly used in settings where the goal is prediction, and one wants
to estimate how accurately a predictive model will perform in practice
Q39. What is Linear regression?
A. A linear regression is a good tool for quick predictive analysis: for
example, the price of a house depends on a myriad of factors, such as its size
or its location. In order to see the relationship between these variables, we
need to build a linear regression, which predicts the line of best fit between
them and can help conclude whether or not these two factors have a positive or
negative relationship.
Q40. What are
the assumptions required for linear regression?
A. There are four major assumptions:
There is a linear
relationship between the dependent variables and the regressors, meaning the
model you are creating actually fits the data
The errors or residuals of the data are
normally distributed and independent from each other
There is minimal multi-co linearity between
explanatory variables
Homoscedasticity. This means the variance
around the regression line is the same for all values of the predictor
variable.
Q41. What is Multiple Regression?
A. Multiple regression generally explains the
relationship between multiple independent or predictor variables and one
dependent or criterion variable. A dependent variable is modeled as a
function of several independent variables with corresponding coefficients,
along with the constant term. Multiple regression requires two or more
predictor variables, and this is why it is called multiple regression.
Q42. What is a Statistical Interaction?
A. Basically, an interaction is when the effect of one factor (input variable)
on the dependent variable (output variable) differs among levels of another
factor.
Q43. What is an example of a data set with a
non-Gaussian distribution?
A.The Gaussian distribution is
part of the Exponential family of distributions, but there are a lot more of
them, with the same sort of ease of use, in many cases, and if the person doing
the machine learning has a solid grounding in statistics, they can be utilized
where appropriate.
Q44. Define Correlation.
A. Correlation is a statistical technique
that can show whether and how strongly pairs of variables are related.
For example: height and weight are related;
taller people tend to be heavier than shorter people. The relationship isn’t
perfect. People of the same height vary in weight, and you can easily think of
two people you know where the shorter one is heavier than the taller one.
Nonetheless, the average weight of people 5’5” is less than the average weight
of people 5’6”, and their average weight is less than that of people 5’7”,
etc.
Correlation can tell you just how much of
the variation in peoples’ weights is related to their heights.
Q45. What is primary goal of A/B Testing?
A. A/B
testing refers to a statistical hypothesis with two variables A and B. The
primary goal of A/B testing is the identification of any changes to the web
page for maximizing or increasing the outcome of interest. A/B testing is a
fantastic method for finding the most suitable online promotional and marketing
strategies for the business.
Q46. What is meaning of Statistical Power
of Sensitivity?
A. The
statistical power of sensitivity refers to the validation of the accuracy of a
classifier, which can be Logistic, SVM, Random Forest, etc. Sensitivity is
basically Predicted True Events/Total Events.
Q47. Explain Over-fitting.
A. In
the case of over-fitting, the model is highly complex, like having too many
parameters which are relative to many observations. The overfit model has poor
predictive performance, and it overreacts to many minor fluctuations in the
training data.
Q48. Explain Under-fitting
A. In the case of under-fitting, the underlying trend of the data
cannot be captured by the statistical model or even the machine learning
algorithm. Even such a model has poor predictive performance.
Q49. What is Long Format Data?
A. In the long format,
every row makes a one-time point per subject. The data in the wide format can
be recognized by the fact that the columns are basically represented by the
groups.
Q50. What is Wide Format Data?
A. In
the wide format, the repeated responses of the subject will fall in a single
row, and each response will go in a separate column.
You can assume anything and everything under the Sun, just to try to keep the assumptions close to reality I always start with an equation, for this question the equation which I assumed was:-
Amount per day * Number of Calendar Days (365)
Assumption 1- A beggar begs all day of the year Now, I have divided a complete day in 4 parts 6 am to 10 am – High income 10 am to 4 pm – Low income 4 pm to 10 pm – High income 10 pm to 6 am – No income
Assumption 2 – The beggar will bet more money in slot 1 and 3 Assumption 3 – Beggar interacts with 500 people in each slot Assumption 4 – The success ratio table
Slot
Success Rate
Number of people giving money
6 AM – 10 AM
0.03
45
10 AM – 4 PM
0.01
15
4 PM – 10 PM
0.05
75
10 PM – 6 AM
0.006
9
144
Assumption 5 – Probability of amount, I have safely assumed that 30% people will give Rs.2, 20% will give Rs.5 and 50% will give Rs.1
Slot
Success Rate
Number of people giving money
Amount
6 AM – 10 AM
0.03
45
94.5
10 AM – 4 PM
0.01
15
31.5
4 PM – 10 PM
0.05
75
157.5
10 PM – 6 AM
0.006
9
18.9
144
302.4
Now we have Rs.302.4 per day income. Annual amount = 302.4*365 = Rs. 110,376 It doesn’t matter if the amount is high or low, what matters is that you have an approach to solve the problem. Few more things which you can add here are:- 1. Divide the year into seasons 2. Divide year into weekend and weekdays 3. Public Holidays
