Recent Discussions
Increasing the number of scripts for Bicep deployments
We were quite surprised to run into a limit of 50 scripts per ADX cluster limit... Considering there is a limit of 10,000 databases, having a limit of 50 scripts seems wrong... Is this a bug? Anyway to increase it? Our use-case would really require that we can provision certain assets alongside our Bicep infra code. We give each customer their own database and provision certain defaults tables and table mapping assets there. We started to hit this 50 script limits just after 4 customers...
Data flow sink to Blob storage not writing to subfolder
Hi Everybody This seems like it should be straightforward, but it just doesn't seem to work... I have a file containing JSON data, one document per line, with many different types of data. Each type is identified by a field named "OBJ", which tells me what kind of data it contains. I want to split this file into separate files in Blob storage for each object type prior to doing some downstream processing. So, I have a very simple data flow - a source which loads the whole file, and a sink which writes the data back to separate files. In the sink settings, I've set the "File name option" setting to "Name file as column data" and selected my OBJ column for the "Column Data", and this basically works - it writes out a separate file for each OBJ value, containing the right data. So far, so good. However, what doesn't seem to work is the very simplest thing - I want to write the output files to a folder in my Blob storage container, but the sink seems to completely ignore the "Folder path" setting and just writes them into the root of the container. I can write my output files to a different container, but not to a subfolder inside the same container. It even creates the folder if it's not there already, but doesn't use it. Am I missing something obvious, or does the "Folder path" setting just not work when naming files from column data? Is there a way around this?September 2025 Recap: What’s New with Azure Database for PostgreSQL
September 2025 Recap for Azure Database for PostgreSQL September was a big month for Azure Postgres! From the public preview of PostgreSQL 18 (launched same day as the community!) to the GA of Azure Confidential Computing and Near Zero Downtime scaling for HA, this update is packed with new capabilities that make PostgreSQL on Azure more secure, performant, and developer-friendly. 💡 Here’s a quick peek at what’s inside: PostgreSQL 18 (Preview) – early access to the latest community release on Azure Near Zero Downtime Scaling (GA) – compute scaling in under 30 seconds for HA servers Azure Confidential Computing (GA) – hardware-backed data-in-use protection PostgreSQL Discovery & Assessment in Azure Migrate (Preview) – plan your migration smarter LlamaIndex Integration – build AI apps and vector search using Azure Postgres VS Code Extension Enhancements – new Server Dashboard + Copilot Chat integration Catch all the highlights and hands-on guides in the full recap 👉 #PostgreSQL #AzureDatabase #AzurePostgres #CloudDatabases #AI #OpenSource #MicrosoftPostgreSQL 18 Preview on Azure Database for PostgreSQL
PostgreSQL 18 Preview on Azure Postgres Flexible Server We’re excited to bring the latest Postgres innovations directly into Azure. With PG18 Preview, you can already test: 🔹 Asynchronous I/O (AIO) → faster queries & lower latency 🔹 Vacuuming enhancements → less bloat, fewer replication conflicts 🔹 UUIDv7 support → better indexing & sort locality 🔹 B-Tree skip scan → more efficient use of multi-column indexes 🔹 Improved logical replication & DDL → easier schema evolution across replicas And that’s just the start — PG18 includes hundreds of community contributions, with 496 from Microsoft engineers alone 💪 👉 Try it out today on Azure Postgres Flexible Server (initially in East Asia), share your feedback, and help shape GA.
Problem with Linked Service to SQL Managed Instance
Hi I'm trying to create a linked Service to a SQL Managed Instance. The Managed Instance is configured with a Vnet_local endpoint If I try to connect with an autoresolve IR or a SHIR I get the following error The value of the property '' is invalid: 'The remote name could not be resolved: 'SQL01.public.ec9fbc2870dd.database.windows.net''. The remote name could not be resolved: 'SQL01.public.ec9fbc2870dd.database.windows.net' Is there a way to connect to it without resorting to a private endpoint? Cheers AlexAzure SQL server rollback itself?
We have an Azure SQL server. It is a datasource of a Power App canvas app. Today I connected to it with SSMS v19. First, I ran 'Begin tran' twice (is it a mistake?). Then 'Delete From dbo.table1 where ID=30' and another row with ID=31. Then I verified these 2 rows are deleted by 'Select * from dbo.table1' Finally, I ran 'Commit tran' I verified again above 2 rows are deleted by 'Select * from dbo.table1' However, there is no change in the Power App. So I reopen the SSMS and connect to the DB again. This time when I ran 'Select * from dbo.table1', the 2 rows are showing up. What could be the problem? Is it a bug in old version SSMS?
ADF connection issue with Cassandra
Hi, I am trying to connect a cassandra DB hosted in azure cosmos db. I created the linked service but getting below error on test connection. Already checked the cassandra DB and its public network access is set to all networks. Google suggested enabling SSL but there is no such option in linked service. Please help. Failed to connect to the connector. Error code: 'Unknown', message: 'Failed to connect to Cassandra server due to, ErrorCode: InternalError' Failed to connect to the connector. Error code: 'InternalError', message: 'Failed to connect to Cassandra server due to, ErrorCode: InternalError' Failed to connect to Cassandra server due to, ErrorCode: InternalError All hosts tried for query failed (tried 51.107.58.67:10350: SocketException 'A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied')August 2025 Recap: Azure Database for PostgreSQL
Here’s what’s new this month to help you build smarter and scale securely: Advisor performance tuning (GA): New insights on index scans, logging, stats, and connections Entra ID group login (Preview): Let users sign in with their own credentials (no need for login using group-ID). New region – Austria East: Lower latency + data residency options for Central Europe LangChain & LangGraph support: Use Azure PostgreSQL as a vector store for AI agents Active-active replication guide: Step-by-step walkthrough using pglogical Full details in monthly recap: https://techcommunity.microsoft.com/blog/adforpostgresql/august-2025-recap-azure-database-for-postgresql/4450527Help with Partial MongoDB Update via Azure Data Factory Data Flow
Hello, everyone! I have a complex question about how to perform a partial update on a MongoDB collection using Data Flow in Azure Data Factory. My goal is to modify only some nested fields without overwriting the entire document. My flow reads JSON files with the following structure: { "_id": { "$oid": "1xp3232to" }, "root_field": "root_value", "main_array": [ { "array_id": "id001", "status": "PENDING", "nested_array": [] } ], "numeric_value": { "$numberDecimal": "10.99" } } I need Data Flow to make two changes in a single run: Change the status field from "PENDING" to "SENT". Add a new object to the nested_array with the following data: event: "SENT" description: "FILE GENERATED" timestamp: (current date and time) system: "Sis Test" I've tried some expressions with update and append in the Derived Column transformation, but I can't get the syntax right to make both changes at the same time. My biggest concern is with the MongoDB Sink: how to configure it so that Data Flow performs a partial update and doesn't overwrite the entire document, losing root_field, numeric_value, etc.? My questions are: What is the correct expression for the Derived Column that makes these two nested modifications in a single step? How should I configure the MongoDB Sink to ensure the update is partial, using _id as the key? I really appreciate the community's help!
Reliable Interactive Resources for Dp300 exam
Hello everyone, I hope you're all having a great day! I wanted to reach out and start a discussion about preparing for the DP-300 (Azure Database Administrator) certification exam. I’ve been researching various resources, but I’m struggling to find reliable and interactive materials that truly help with exam prep. For those who have already passed the DP-300, could you share any interactive and trustworthy resources you used during your study? Whether it's courses, hands-on labs, or practice exams, I’d really appreciate your recommendations. Any advice on how to effectively prepare would be incredibly helpful! Thank you so much for your time reading this discussion and for sharing your experiences!Prevent Accidental Deletion of an Instance in Azure Postgres
Did you know that accidental deletion of database servers is a leading source of support tickets? Read this blog post to learn how you can safeguard your Azure Database for PostgreSQL Flexible Server instances using ARM’s CanNotDelete lock — an easy best-practice that helps prevent accidental deletions while keeping regular operations seamless. 🌐 Prevent Accidental Deletion of an Instance in Azure Postgrestimechart legend in Azure Data Explorer
Hi, I'm creating a timechart dashboard in Azure Data Explorer and facing an issue with the legend labels. The legends have extra prefixes and suffixes, such as "Endpoint" or "Count". How can I remove these and show only the actual value in the legend? Thank you!
User configuration issue
Hi, I am getting below error "Execution fail against sql server. Please contact SQL Server team if you need further support. Sql error number: 3930. Error Message: The current transaction cannot be committed and cannot support operations that write to the log file. Roll back the transaction" I never faced this kind of error. Could anyone please let me know what i can do and i needs to do. I am beginner, please explain me. Thanks.How to use existing cache for external table when acceleration in progress
I enabled query acceleration for my external table which binds a delta table (1TB) on ADLS. But acceleration progress needs 1.5 hours to complete. I found during acceleration in progress, querying on the table is quite slower than case when acceleration completed. How can I use the existing acceleration cache/index and after acceleration completed, Kusto will switch to new index?
Getting an Oauth2 API access token using client_id and client_secret - help
Hi, I'm attempting to integrate external data into our SQL Server. The third-party data is from a solution called iLevel. They use token based Oauth2 APIs for access. The integration tool is ADF Pipelines. I'm not a data engineer but it has fallen upon me to complete this exercise. What I've attempted so far is failing and I don't know why. I would like your help on this. I'll explain what I've configured so far in the order I configured it. 1) To generate a client_id and client_secret, I logged on to the iLevel solution itself and generated the same for my account (call me 'Joe' account) and the Team account (call it 'Data team' account). I've recorded the client_id and client_secret for both users/accounts in notepad for reference. 2) I logged in Azure Data Factory using my 'Joe Admin' admin account (this is the account I need to log in with for any ADF development). 3) I created a Linked Service with the following configuration. Note how the Test connection was successful. I guess this means our ADF instance can connect to iLevel's Base URL. 4) I then created a dataset for iLevel. I configured this based on an online example I was following which I can't get working, so this configuration may be incorrect. 5) I then created a Pipeline which contains a 'Web' activity and a 'Set variable' activity. The Pipeline has a variable as shown below. The 'Web' activity has the following configuration: URL = is iLevel's token URL (it is different from the Base URL used in the Linked Service). Body = I've blocked out the client_id and client_secret (I'm using the client_id and client_secret generated for the 'Data team' account - remember I'm logged into ADF using the 'Joe Admin' account - not sure if this makes a difference) but have placed red brackets around where the start and end of each values is. I'm wrapping the values in any single or double quotes - not sure if I'm meant to. I'm not sure if I have configured the Body correctly. The ilevel documentation states to use an Authorization header, Content-Type header and Body - it states to the following is needed to obtain an access token, but it doesn't state exactly how to submit the information (i.e. how to format it). Notice how, in my configuration, I haven't used an Authorization header - this partly because an online example I've followed doesn't use one. If iLevel state to use one then I think I should but I don't know how to format it - any ideas? The 'Set variable' activity has the following activity. The idea is the access token is retrieved from the 'Web' activity and placed in the 'Set variable' "iLevel access token" variable. At this point I validate all and it comes back with no errors found. I then Debug it to see if it does indeed work but it returns an error stating the request contains an invalid client_id or client_secret. The client_id and client_secret values used are the exact same I generated from within the iLevel solution just a few hours ago. Is anyone able to point out to me why this isn't working? Have I populated all that I need to (as mentioned, iLevel say to use an Authorization header which I haven't but I don't know how to format it if I were to use one)? What can I do to get this working? I'm just trying to get the access token at the moment. I've not even attempted to extract the iLevel data and can't until I get a working token. iLevel's token have a 1 hour time-to-live so the Pipeline needs to generate a new token each time it's executed. You help will be most appreciated. Thanks.Best practice to integrate to Azure DevOps?
Different sources suggesting different recommendations regarding ADF and ADO integration. Some say to use 'adf_publish' branch, while some suggest to use 'main' branch to be source for triggering yaml pipelines and disabling 'Publish' function in ADF. I guess practices are changing and setup could be different. The problem is finding all this information on the Internet makes it so confusing. So, the question is what is the best practice now (taking into account all the latest changes in ADO) regarding branches? How you set up your ADF and ADO integrations?July 2025 Recap: Azure Database for PostgreSQL
Azure PostgreSQL – July 2025 Recap is live! This month’s update brings a strong mix of performance, security, and developer experience improvements — plus new region rollouts and deeper analytics integrations. What’s new: Cascading Read Replica (Preview) for multi-level replication Fabric Mirroring with VNET + Private Endpoint (Preview) Power BI Entra ID authentication (GA) Upsert + Script activity in ADF/Synapse (GA) Now available in Malaysia West & Chile Central PostgreSQL minor version updates: 17.5, 16.9, 15.13, 14.18, 13.21 Agentic Web + NLWeb support for PostgreSQL VS Code extension enhancements Improved maintenance for stopped instances Read the complete blog post - Azure PostgreSQL – July 2025 Recap is live!Azure SQL Firewall / Locks
Hi there, I have 2 environments. I'm more of admin on Azure environment (recently made as subscription admin) after which Dev issue - Azure SQL I'm having difficulty to remove IP from Azure SQL Firewall. (Earlier i was able to) today my manager granted me subscription admin and as SQL Security Manager and it still not able to remove grayed out IPs.
Recent Blogs
- Azure SQL Managed Instance improves and promotes "Redirect" to become the default connection type. Redirect now only requires port 1433 and can handle older drivers gracefully.Oct 22, 2025
- Errors are an inherent part of any application. As a database professional managing an Azure SQL Managed Instance, you may be interested in understanding when specific errors occur and how to leverag...Oct 21, 2025
