<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" > <channel> <title> Comments on: How To Scale Azure SQL Database In Azure Data Factory Using A Web Activity </title> <atom:link href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/feed/" rel="self" type="application/rss+xml" /> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/?utm_source=rss&utm_medium=rss&utm_campaign=scale-azure-sql-using-adf-web-activity</link> <description>A blog on Power BI & Azure Data Platform</description> <lastBuildDate>Wed, 13 Jul 2022 03:42:40 +0000</lastBuildDate> <sy:updatePeriod> hourly </sy:updatePeriod> <sy:updateFrequency> 1 </sy:updateFrequency> <generator>https://wordpress.org/?v=6.1.7</generator> <item> <title> By: Kumar </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-9631</link> <dc:creator><![CDATA[Kumar]]></dc:creator> <pubDate>Wed, 13 Jul 2022 03:42:40 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-9631</guid> <description><![CDATA[Hi Dave, I am getting the below error message. Can you please help me out? {"error":{"code":"ParentResourceNotFound","message":"Can not perform requested operation on nested resource. Parent resource 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' not found."}}]]></description> <content:encoded><![CDATA[<p>Hi Dave, </p> <p>I am getting the below error message. Can you please help me out?</p> <p>{“error”:{“code”:”ParentResourceNotFound”,”message”:”Can not perform requested operation on nested resource. Parent resource ‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’ not found.”}}</p> ]]></content:encoded> </item> <item> <title> By: Dave Ruijter </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-9133</link> <dc:creator><![CDATA[Dave Ruijter]]></dc:creator> <pubDate>Fri, 24 Jun 2022 05:00:53 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-9133</guid> <description><![CDATA[In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-8904">Robert</a>. Always good to apply least privilege <img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f44d-1f3fb.png" alt="👍🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />]]></description> <content:encoded><![CDATA[<p>In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-8904">Robert</a>.</p> <p>Always good to apply least privilege 👍🏻</p> ]]></content:encoded> </item> <item> <title> By: Robert </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-8904</link> <dc:creator><![CDATA[Robert]]></dc:creator> <pubDate>Wed, 15 Jun 2022 22:52:33 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-8904</guid> <description><![CDATA[Hi Dave, I have this set up, however the higher ups at my company feel that Contributor may be too permissive. They even feel that SQL DB Contributor may still be too permissive. I'm thinking a custom RBAC role with the following permissions may work. Read List/Get Azure SQL Database(s) "Microsoft.Sql/servers/databases/read" Write Create/Update Azure SQL Database "Microsoft.Sql/servers/databases/write" Other Pause a Datawarehouse database. "Microsoft.Sql/servers/databases/pause/action" Other Resume a Datawarehouse database. "Microsoft.Sql/servers/databases/resume/action" Read List/Get Azure SQL Server(s) "Microsoft.Sql/servers/read" Read Gets the status of a database operation "Microsoft.Sql/servers/databases/operationResults/read" Can you provide any feedback on this?]]></description> <content:encoded><![CDATA[<p>Hi Dave,<br /> I have this set up, however the higher ups at my company feel that Contributor may be too permissive. They even feel that SQL DB Contributor may still be too permissive. I’m thinking a custom RBAC role with the following permissions may work. </p> <p>Read List/Get Azure SQL Database(s) “Microsoft.Sql/servers/databases/read”<br /> Write Create/Update Azure SQL Database “Microsoft.Sql/servers/databases/write”<br /> Other Pause a Datawarehouse database. “Microsoft.Sql/servers/databases/pause/action”<br /> Other Resume a Datawarehouse database. “Microsoft.Sql/servers/databases/resume/action”<br /> Read List/Get Azure SQL Server(s) “Microsoft.Sql/servers/read”<br /> Read Gets the status of a database operation “Microsoft.Sql/servers/databases/operationResults/read”</p> <p>Can you provide any feedback on this?</p> ]]></content:encoded> </item> <item> <title> By: David </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3935</link> <dc:creator><![CDATA[David]]></dc:creator> <pubDate>Thu, 06 May 2021 17:01:38 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3935</guid> <description><![CDATA[Thank you sir! that worked like a charm !]]></description> <content:encoded><![CDATA[<p>Thank you sir! that worked like a charm !</p> ]]></content:encoded> </item> <item> <title> By: Dave Ruijter </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3845</link> <dc:creator><![CDATA[Dave Ruijter]]></dc:creator> <pubDate>Wed, 21 Apr 2021 20:03:20 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3845</guid> <description><![CDATA[In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3773">Bas</a>. Hi Bas, Sorry I missed your questions. By default, the web activity in Azure Data Factory has a timeout of 1min for POST requests. So that is what is causing this issue. And, it seems that this POST request is run synchronous, so it waits until it completes.. Maybe you can think of a solution that is asynchronous?]]></description> <content:encoded><![CDATA[<p>In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3773">Bas</a>.</p> <p>Hi Bas,</p> <p>Sorry I missed your questions.<br /> By default, the web activity in Azure Data Factory has a timeout of 1min for POST requests.<br /> So that is what is causing this issue.<br /> And, it seems that this POST request is run synchronous, so it waits until it completes..<br /> Maybe you can think of a solution that is asynchronous?</p> ]]></content:encoded> </item> <item> <title> By: Bas </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3773</link> <dc:creator><![CDATA[Bas]]></dc:creator> <pubDate>Mon, 12 Apr 2021 14:45:18 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3773</guid> <description><![CDATA[Hi Dave, this is an addition to my previous message (can't respond to that one yet, still waiting moderation). It now seems that sometimes the pipeline works, sometimes it fails. I can imagine that for larger databases this operation will fail more often - as Microsoft are replicating your database (which takes time depending on data volume) in a different service tier when you're scaling the DB.]]></description> <content:encoded><![CDATA[<p>Hi Dave, this is an addition to my previous message (can’t respond to that one yet, still waiting moderation).<br /> It now seems that sometimes the pipeline works, sometimes it fails. I can imagine that for larger databases this operation will fail more often – as Microsoft are replicating your database (which takes time depending on data volume) in a different service tier when you’re scaling the DB.</p> ]]></content:encoded> </item> <item> <title> By: Bas </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3772</link> <dc:creator><![CDATA[Bas]]></dc:creator> <pubDate>Mon, 12 Apr 2021 14:33:06 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3772</guid> <description><![CDATA[Hi Dave, this looked very promising when I tried it out and it worked immediately. But when I trigger the pipeline the web activity times out after 1 min. Have you seen that issue before, and maybe can you suggest a solution?]]></description> <content:encoded><![CDATA[<p>Hi Dave, this looked very promising when I tried it out and it worked immediately. But when I trigger the pipeline the web activity times out after 1 min. Have you seen that issue before, and maybe can you suggest a solution?</p> ]]></content:encoded> </item> <item> <title> By: Dave Ruijter </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3634</link> <dc:creator><![CDATA[Dave Ruijter]]></dc:creator> <pubDate>Fri, 19 Mar 2021 11:42:51 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3634</guid> <description><![CDATA[In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3633">bhaskar</a>. Yes, that should be possible with some minor adjustments]]></description> <content:encoded><![CDATA[<p>In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3633">bhaskar</a>.</p> <p>Yes, that should be possible with some minor adjustments</p> ]]></content:encoded> </item> <item> <title> By: bhaskar </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-3633</link> <dc:creator><![CDATA[bhaskar]]></dc:creator> <pubDate>Fri, 19 Mar 2021 11:25:54 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-3633</guid> <description><![CDATA[Can we do the same for Azure Sql POOL]]></description> <content:encoded><![CDATA[<p>Can we do the same for Azure Sql POOL</p> ]]></content:encoded> </item> <item> <title> By: Dave Ruijter </title> <link>https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-2193</link> <dc:creator><![CDATA[Dave Ruijter]]></dc:creator> <pubDate>Thu, 15 Oct 2020 10:13:36 +0000</pubDate> <guid isPermaLink="false">https://www.moderndata.ai/?p=223#comment-2193</guid> <description><![CDATA[In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-2155">Jos Haemers</a>. You can change the tier to get more DTUs. You can't set the DTU level.]]></description> <content:encoded><![CDATA[<p>In reply to <a href="https://www.moderndata.ai/2018/12/scale-azure-sql-using-adf-web-activity/#comment-2155">Jos Haemers</a>.</p> <p>You can change the tier to get more DTUs. You can’t set the DTU level.</p> ]]></content:encoded> </item> </channel> </rss>