Click here to Skip to main content
15,607,903 members
Home / Discussions / Cloud Computing
   

Cloud Computing

 
QuestionHow to set up Minikube and get started with kubernetes? Pin
MoistMan20217-Mar-23 18:50
MoistMan20217-Mar-23 18:50 
QuestionLogging in to Tiki Wiki on Azure Pin
jsc427-Jan-22 5:29
professionaljsc427-Jan-22 5:29 
QuestionCan a SignalR chat "hub" be hosted on AWS/LightSail? Pin
victorbos27-Jan-21 4:14
victorbos27-Jan-21 4:14 
AnswerMessage Closed Pin
15-Mar-21 4:06
Jhonalbert15-Mar-21 4:06 
AnswerRe: Can a SignalR chat "hub" be hosted on AWS/LightSail? Pin
Leeladhar Ladia25-Mar-22 3:11
Leeladhar Ladia25-Mar-22 3:11 
QuestionSFC allocation in CloudSim Pin
Member 147774293-Jan-21 1:30
Member 147774293-Jan-21 1:30 
QuestionWhich Cloud vendor chooses for the service website? Pin
William.Front-end3-Sep-20 0:48
William.Front-end3-Sep-20 0:48 
AnswerRe: Which Cloud vendor chooses for the service website? Pin
Afzaal Ahmad Zeeshan3-Sep-20 5:13
professionalAfzaal Ahmad Zeeshan3-Sep-20 5:13 
GeneralRe: Which Cloud vendor chooses for the service website? Pin
William.Front-end8-Sep-20 0:47
William.Front-end8-Sep-20 0:47 
AnswerRe: Which Cloud vendor chooses for the service website? Pin
ZurdoDev15-Oct-20 6:45
professionalZurdoDev15-Oct-20 6:45 
AnswerRe: Which Cloud vendor chooses for the service website? Pin
MobilHead6-Aug-21 1:08
MobilHead6-Aug-21 1:08 
AnswerRe: Which Cloud vendor chooses for the service website? Pin
Michael Donald29-Nov-21 21:08
professionalMichael Donald29-Nov-21 21:08 
AnswerRe: Which Cloud vendor chooses for the service website? Pin
Member 155103601-Jun-22 4:56
Member 155103601-Jun-22 4:56 
QuestionMigrate shared hosting ASP.Net site(s) to Azure Pin
DerekT-P18-Jul-20 23:28
professionalDerekT-P18-Jul-20 23:28 
AnswerRe: Migrate shared hosting ASP.Net site(s) to Azure Pin
Afzaal Ahmad Zeeshan3-Sep-20 5:24
professionalAfzaal Ahmad Zeeshan3-Sep-20 5:24 
GeneralRe: Migrate shared hosting ASP.Net site(s) to Azure Pin
DerekT-P3-Sep-20 23:41
professionalDerekT-P3-Sep-20 23:41 
GeneralRe: Migrate shared hosting ASP.Net site(s) to Azure Pin
Afzaal Ahmad Zeeshan4-Sep-20 3:57
professionalAfzaal Ahmad Zeeshan4-Sep-20 3:57 
QuestionOffice 365 + IMAP + OAuth2 [Solved] Pin
Richard Deeming15-Apr-20 4:03
mveRichard Deeming15-Apr-20 4:03 
AnswerRe: Office 365 + IMAP + OAuth2 Pin
Richard Deeming20-Apr-20 0:41
mveRichard Deeming20-Apr-20 0:41 
AnswerRe: Office 365 + IMAP + OAuth2 Pin
Richard Deeming28-Apr-20 0:21
mveRichard Deeming28-Apr-20 0:21 
QuestionASP.NET + Multi Form using Textbox problem Pin
kerek224-Feb-20 21:42
kerek224-Feb-20 21:42 
AnswerRe: ASP.NET + Multi Form using Textbox problem Pin
Richard MacCutchan24-Feb-20 23:17
mveRichard MacCutchan24-Feb-20 23:17 
QuestionAzure Data Factory pipeline into compressed Parquet file: “java.lang.OutOfMemoryError:Java heap space” Pin
VictorSotnikov16-Sep-19 10:20
VictorSotnikov16-Sep-19 10:20 
I have met the following problem. I have a pipeline that reads data from an MS SQL Server and stores them into a file in a BLOB container in Azure Storage. The file has Parquet (or Apache Parquet, as it is also called) format.
So, when the “sink” (output) file is stored in a compressed way (snappy, or gzip – does not matter) AND the file is large enough (more than 50 Mb), the pipeline failed. The message was the following:
"errorCode": "2200",
"message": "Failure happened on 'Sink' side. ErrorCode=UserErrorJavaInvocationException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking java, message: java.lang.OutOfMemoryError:Java heap space\ntotal entry:11\r\njava.util.ArrayDeque.doubleCapacity(Unknown Source)\r\njava.util.ArrayDeque.addFirst(Unknown Source)\r\njava.util.ArrayDeque.push(Unknown Source)\r\norg.apache.parquet.io.ValidatingRecordConsumer.endField(ValidatingRecordConsumer.java:108)\r\norg.apache.parquet.example.data.GroupWriter.writeGroup(GroupWriter.java:58)\r\norg.apache.parquet.example.data.GroupWriter.write(GroupWriter.java:37)\r\norg.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:87)\r\norg.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:37)\r\norg.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:123)\r\norg.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:292)\r\ncom.microsoft.datatransfer.bridge.parquet.ParquetBatchWriter.addRows(ParquetBatchWriter.java:60)\r\n,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.DataTransfer.Richfile.JniExt.JavaBridgeException,Message=,Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'",
"failureType": "UserError",
"target": "Work_Work"
}

The "Work_Work" is name of a Copy Data activity in the pipeline.
If I turn the compression off (the generated BLOB file is uncompressed), the error does not happen.
Is this the error described in https://docs.microsoft.com/en-us/azure/data-factory/format-parquet: the “…If you copy data to/from Parquet format using Self-hosted Integration Runtime and hit error saying "An error occurred when invoking java, message: java.lang.OutOfMemoryError:Java heap space", you can add an environment variable _JAVA_OPTIONS in the machine that hosts the Self-hosted IR to adjust the min/max heap size for JVM to empower such copy, then rerun the pipeline….”?

If it is, have I understood correctly that I have to do the following:
To go to a server where the “Self-hosted Integration Runtime” (still have no idea what it is) and increase the max heap size for JVM. Is this correct?
If it is, my next question is: how large the max heap size should be? My pipeline can generate a file whose size will be 30 GB.
What “max heap size” can guarantee that such a file will not cause the fail?
QuestionHow to search for a word or color on a website? Pin
l@rs_30-Aug-19 8:17
l@rs_30-Aug-19 8:17 
AnswerRe: How to search for a word or color on a website? Pin
Afzaal Ahmad Zeeshan30-Aug-19 12:02
professionalAfzaal Ahmad Zeeshan30-Aug-19 12:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.