|
I have been playing around with this for some time and I can't get an elegant solution. My code always looks like I am missing some obvious library.
We have a lot of old fortran code lying about doing Numerical Simulations. I am trying to get some movement into a sensible coding environment. It is becoming prohibitive using Fortran since the skill set (across our company) is diminishing and therefore becoming expensive.
Step one:
Recompile code using the SilverFrost compiler using the .NET options.
Step two:
Call Fortran code from C#.
Step Three:
Start putting result from Fortran into a biggish Sql say 5GB per model run od so
I have got something sensible for steps 1 and 2 and general movement is going in the write direction, step 3 seem to be convoluted.
say I have 1 dimenesional array of doubles, I simply want to record in the DB
something like
Index,Answer
0,0.1
1,0.2
2,0.3
3,0.4
4,0.5
5,0.6
6,0.7
7,0.9
currently the Fortran just writes CSV files in the above format.
We could bulk copy the CSV files into the DB but seems like an unnessary step (the CSV file), why not direct?.
We could convert the Arrays into DataTables and use SqlBulkCopy,
( just making the DataTable seems inelegant )
We could write a custom IDataReader for arrays and use SqlBulkCopy,
(Idatareaders seems to require an enormous amount of unsued methods to implemnt)
SqlBulkCopy is not an option for things like Access though or SqlCe if we wanted to use a similar methodolgy for smaller localised models
So the question is whats the "best" way of getting a large .net array into SqlCe,SQL and Access datbases.
I guess thats 3 questions one for each format
Carl
DrTip
|
|
|
|
|
You can either do this using an SSIS package (SQL SErver only), or do something closer to this[^], which would be more a more generic approach.
|
|
|
|
|
Thank Dave
I am aware of both of these methods. But I think you have missed the point of the question
The thing we want is to write lots of data to a database directly from memory without first writing to an intermediate text file.
The data is generated from a numerical model that typically takes 18 Hours to run.
Carl
Quick the boss is coming...
|
|
|
|
|
Well, you said it yourself - the FORTRAN code wirte this stuff to a CSV file currently. Your ONLY option is to rewrite the FORTRAN code to NOT write it to a file, but instead open it's own connection to the database and start writing it there itself, as the data is generated, one record at a time. This is because of the lack of a bulk copy in anything other that SQL Server.
modified on Thursday, January 10, 2008 4:23:15 PM
|
|
|
|
|
I was going to ask this in the Vista forum but that appeared to refer more to the Vista win API than to how the .NET framework runs on Vista, so I'm asking here..
Maybe my Google-fu sucks, but I can't seem to find any pointers on how to write applications to ensure they'll run on Vista without elevated privileges.
In particular, I'm interested in the following things that will trigger the elevated permissions system:
* Which P/Invoke functions to avoid
* Which .NET namespaces to avoid (the Cryptography namespace forces elevated privileges?!?!?!?!?!?!)
* Which files and directories to avoid (I know, \Program Files\ and %WINDIR%\system. Any others?)
Also, a bonus would be some kind of tool that will tell me specifically which of these areas will cause Vista to refuse to run my application. I found this tool[^], but all that does is report errors from the low-level operating system API that mean very little to me.
Fundamentally my issue is that I'm writing an application that RUNS fine on XP, INSTALLS fine on Vista using a VS2005 setup project, (but it DOES request elevated permissions), but it will not RUN on Vista (UAC does ask to allow the program to run). And even when the user selects ALLOW, the program still crashes.
For the life of me I can't seem to pinpoint why. The best I seem to be able to find are recommendations of embedding a manifest file to demand admin privileges. Well, that isn't what I want to do. I want my application to run under normal user permissions, but I can't figure out WHAT in my application is triggering the request to elevated privileges.
Pointers are appreciated... this is pretty aggravating.
It has become appallingly obvious that our technology has exceeded our humanity. - Albert Einstein
modified on Monday, January 07, 2008 4:03:11 PM
|
|
|
|
|
|
Thanks Mark, I'll take a look. I also found a 30-day eval Vista virtual hard drive, so I will do some testing.
It has become appallingly obvious that our technology has exceeded our humanity. - Albert Einstein
|
|
|
|
|
Don't think of what to avoid, think of what the minimum you should do is. It's safe to write to the user's profile. That's pretty much it. Anything that changes the system configuration is likely to fail for a standard/non-elevated user.
No APIs force elevated privileges. UAC will automatically apply elevation if it thinks that you're a legacy setup program (if your program name or resources contain the words 'setup' or 'install') or if there's a compatibility shim detecting the program that requires it to run elevated. Otherwise, your program does not prompt, and runs in a 'virtualized' environment, where writes to certain registry keys and protected disk locations are redirected to shadow folders under the user profile.
A manifest is necessary to tell Windows Vista that you are aware of the presence of UAC. If you never want it to prompt, set the requestedExecutionLevel to asInvoker. This means 'never ask for administrative rights even if the user is an admin'. The other available settings are requireAdministrator, which always prompts (for confirmation for administrators, for admin credentials for non-admins), and highestAvailable, which prompts administrators for confirmation but runs without admin privileges, without prompting, for non-administrators. The latter is a bit of a compromise and you're recommended to strip the administrative functions out into a separate program.
As soon as you install a manifest containing one of these settings, Windows Vista will disable the legacy redirection. You'll then get exceptions as appropriate as you use APIs for which you don't have permission.
Another simple way to test is to run your program under Windows XP as a standard user. That'll blow up all over the place.
The Windows security model is so much more sophisticated than a simple admins can/non-admins can't system. Most of Windows security is based on Access Control Lists, which define exactly what different security principals (users or groups) can or can't do. In addition there are a separate set of privileges which are again assigned to users or groups; these privileges largely allow you to override the ACLs (for example, the ability to change the ACL is normally a permission in the ACL, but the owner of the object can overwrite it even if this permission is denied to them in the ACL; administrators also have the right to Take Ownership of the object, then they can write to the ACL).
DoEvents: Generating unexpected recursion since 1991
|
|
|
|
|
Mike Dimmick wrote: Another simple way to test is to run your program under Windows XP as a standard user. That'll blow up all over the place.
Did that.. didn't reveal the issues that came up on Vista though.
That said, a manifest both in the installer and in my executable appears to have resolved everything.
Incidentally, the installer (the bootstrapper, actually) needs its requestedExecutionLevel set to requireAdministrator in order for installs for Standard Users to work correctly. I'm not sure why that is - if it is set to asInvoker, when the Standard User goes to use the program for the first time and the installer does the shortened install, the files and folders I create in the user's AppData folder don't get created.
If the original installer uses requireAdministrator, however, that works just fine.
Anyway, thanks for your insight 
It has become appallingly obvious that our technology has exceeded our humanity. - Albert Einstein
|
|
|
|
|
I like to use the SerialPort class (namespace System.IO.Ports) in an existing MSVS2003 project (.Net1.1).
Is this possible and how to proceed?
Can anybody give me a hint?
Regards Peter
|
|
|
|
|
Hi, SerialPort is part of .NET since version 2.0, and MSVS2003 only targets .NET 1.1,
so no you can't use it.
You could use MSVS2005 (the Express Editions are freely downloadable) and convert
your project, which typically is straightforward. Major drawback is the target machines
for your app now need .NET 2.0 (which is freely downloadable too, but needs to be installed).
|
|
|
|
|
Thanks for the reaction
Peter
|
|
|
|
|
Hi
My company has been happily bumbling along storing configuration data for our application in the registry and storing any application data in \Program Files\application\data. Unfortunately a Vista laptop has just been presented to me with the request to get one of our applications to work on it.
I think we are going to have to upgrade our data storage policy. Can someone point me at a MS best practice document of what we really should be doing? Or better still something that will work with Mono as well?
Thanks
Dave
|
|
|
|
|
|
Dear all,
I have a question about preserving the backward versioning compatibly in Visual C++ .NET 2003:
I am building a win form control using Visual C++ under Microsoft Visual Studio .NET 2003 framework. I already made a previous release of this control with a version number (1.0.0.0) and I have customers who already built projects using my control. Currently I am working on a new release of this control with a new version number (2.0.0.0) which includes new features and properties (I updated the major and minor version numbers in the assembly file). My issue is that this new release recommend a change in one of the old properties (that existed in version 1.0.0.0) default values, I need to know the correct method of changing the old property default value without affecting the already existed projects that use the old default values. In another words, I need to preserve the backward versioning compatibility.
Please advice..
Thanks in advance..
|
|
|
|
|
I have two DataTable Objects with exactly the same structures. I want to copy one row from one of the tables to the other. And to do this I write this code snippet:
DataTable1.Rows.Add(DataTable2.Rows[0]);
But I get this error "Row belongs to another table". How can I achieve the result ? Any idea ? I need to make an exact copy of a reference type object(like a DataRow object in this example) with addresses different.
|
|
|
|
|
|
I have created a C# application and used threading for creating 400 folders.And I have a picture file to pass to those 400 folders.It took an average of 30 seconds in server part and an average of 10 seconds in client part.
Is it possible to accomplish the task in 1 second.And lets not care the server part and its performance.
Thanx in advance.
|
|
|
|
|
First, is impossible to "not care" the server, because you write a lot of folders and pictures phisically inside the server. Dry Response -> NOT IN 1 SECOND! The times seems to be ok, but maybe is better if you post the code...
|
|
|
|
|
Member 2998912,Thanx for the reply.
"dont care" the server part means that it is okay if server part becomes hang also.Below is the code i used to create 400 folders and threading is also required.
//Method to create folder
int j=0;
public void CreateFolders()
{
Directory.CreateDirectory(@"d:\test1\" + j );
string name = "namobuddha.jpeg";
string filename = @"d:\test1\p\namobuddha.jpeg";
Image im = Image.FromFile(filename);
MemoryStream ms = new MemoryStream();
im.Save(ms, System.Drawing.Imaging.ImageFormat.Gif);
byte[] image1 = ms.ToArray();
//for the method of server part
Service a1 = new Service();
a1.capture2(image1, j.ToString(), name);// method from webservice
j=j+1;
}
public void ForThread()
{
Thread[] t1 = new Thread[400];
for (int i = 0; i <= 399; i++)
{
t1[i] = new Thread(new ThreadStart(CreateFolders));
t1[i].Start();
t1[i].Join();
}
}
Can there be any solution using the TimeOut property in server part or client part.
|
|
|
|
|
|
Thank you Luc Pattyn.
In client part, 400 folders can just be created in one second by using thread or without using it also.but server part can't retrieve all folders in such small time with all folders containing same image.So I have done that way.I dont know whether it is effective or not.but i ll try other ways also.Is there any efficient way that u can suggest me?Can anything be done in the server part?
thank u.
happy 2008.
|
|
|
|
|
|
This code snippet is a little extension to code-project's excellent article on large file transfer using Mtom and streamed mode.
Initially i have created a project in Server 2003 environment and have modified few of my server settings to make the project work.
I wanted to test my work on different environment for demonstration purpose. And i have come across many runtime exceptions and most of them being general exceptions which i also tried to search on web for a solution but could not find right solution.
I think detail description of my appconfig, webconfig, Service Contract would help to guide me better.
AppConfig
<configuration>
<system.servicemodel>
<bindings>
<basichttpbinding>
<binding name="BasicHttpBinding_IFileTransferService" maxreceivedmessagesize="10067108864">
messageEncoding="Mtom" transferMode="Streamed" />
</binding></basichttpbinding>
</bindings>
<client>
<endpoint address="http://localhost:2196/FileTransferService.svc/mex">
binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IFileTransferService"
contract="Proxy.IFileTransferService" name="BasicHttpBinding_IFileTransferService" />
</endpoint></client>
</system.servicemodel>
</configuration>
Webconfig
<configuration>
<appsettings />
<connectionstrings />
<system.web>
<httpruntime maxrequestlength="65536" />
<compilation debug="true"></compilation>
<authentication mode="None" />
</system.web>
<system.servicemodel>
<bindings>
<basichttpbinding>
<binding name="FileTransferServicesBinding">
transferMode="Streamed"
messageEncoding="Mtom"
maxReceivedMessageSize="10067108864">
</binding>
</basichttpbinding>
</bindings>
<services>
<service behaviorconfiguration="MyServiceTypeBehaviors" name="WebHost.FileTransferService">
<endpoint address="mex">
binding="basicHttpBinding"
bindingConfiguration="FileTransferServicesBinding"
contract="WebHost.IFileTransferService" />
</endpoint></service>
</services>
<behaviors>
<servicebehaviors>
<behavior name="MyServiceTypeBehaviors">
<servicemetadata httpgetenabled="true" />
<servicedebug includeexceptiondetailinfaults="true" />
</behavior>
</servicebehaviors>
</behaviors>
</system.servicemodel>
</configuration>
Service Contract
using System;
using System.ServiceModel;
namespace WebHost
{
[ServiceContract]
public interface IFileTransferService
{
[OperationContract]
void UploadFile(RemoteFileInfo request);
[OperationContract]
RemoteFileInfo DownloadFile(DownloadRequest request);
}
[MessageContract]
public class DownloadRequest
{
[MessageBodyMember]
public string FileName;
}
[MessageContract]
public class RemoteFileInfo: IDisposable
{
[MessageHeader(MustUnderstand = true)]
public string FileName;
[MessageHeader(MustUnderstand = true)]
public long Length;
[MessageBodyMember(Order = 1)]
public System.IO.Stream FileByteStream;
public void Dispose()
{
if (FileByteStream!=null)
{
FileByteStream.Close();
FileByteStream = null;
}
}
}
}
The exceptions i get on chaning the environment are...
The message with Action '' cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and receiver) or a binding/security mismatch between the sender and the receiver. Check that sender and receiver have the same contract and the same binding (including security requirements, e.g. Message, Transport, None).
stack call being
Server stack trace:
at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at FileTransferClient.Proxy.IFileTransferService.UploadFile(RemoteFileInfo request)
at FileTransferClient.Proxy.FileTransferServiceClient.FileTransferClient.Proxy.IFileTransferService.UploadFile(RemoteFileInfo request) in C:\Documents and Settings\pkumar\Desktop\CS\CS\FileTransferClient\Service References\Proxy\Reference.cs:line 105
at FileTransferClient.Proxy.FileTransferServiceClient.UploadFile(String FileName, Int64 Length, Stream FileByteStream) in C:\Documents and Settings\pkumar\Desktop\CS\CS\FileTransferClient\Service References\Proxy\Reference.cs:line 113
at FileTransferClient.TestForm.UploadButton_Click(Object sender, EventArgs e) in C:\Documents and Settings\pkumar\Desktop\CS\CS\FileTransferClient\TestForm.cs:line 68
Now on changing the chunksize on the server's service method from 4096 to 2048 i get an exception
The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:59.9840000'.
and stack of errors
Server stack trace:
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.WebRequestOutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.BufferedStream.FlushWrite()
at System.IO.BufferedStream.Flush()
at System.Xml.MimeWriter.Close()
at System.Xml.XmlMtomWriter.WriteXOPBinaryParts()
at System.Xml.XmlMtomWriter.WriteEndElement()
at System.ServiceModel.Channels.Message.OnWriteMessage(XmlDictionaryWriter writer)
at System.ServiceModel.Channels.Message.WriteMessage(XmlDictionaryWriter writer)
at System.ServiceModel.Channels.MtomMessageEncoder.WriteMessage(Message message, Stream stream, String startInfo, String boundary, String startUri, Boolean writeMessageHeaders)
at System.ServiceModel.Channels.HttpOutput.WriteStreamedMessage(TimeSpan timeout)
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at FileTransferClient.Proxy.IFileTransferService.UploadFile(RemoteFileInfo request)
at FileTransferClient.Proxy.FileTransferServiceClient.FileTransferClient.Proxy.IFileTransferService.UploadFile(RemoteFileInfo request) in C:\Documents and Settings\pkumar\My Documents\Visual Studio 2008\Projects\WebHost\FileTransferClient\Service References\Proxy\Reference.cs:line 105
at FileTransferClient.Proxy.FileTransferServiceClient.UploadFile(String FileName, Int64 Length, Stream FileByteStream) in C:\Documents and Settings\pkumar\My Documents\Visual Studio 2008\Projects\WebHost\FileTransferClient\Service References\Proxy\Reference.cs:line 113
at FileTransferClient.TestForm.UploadButton_Click(Object sender, EventArgs e) in C:\Documents and Settings\pkumar\My Documents\Visual Studio 2008\Projects\WebHost\FileTransferClient\TestForm.cs:line 68
Now changing the AppConfig binding to
<bindings>
<basichttpbinding>
<binding name="BasicHttpBinding_IFileTransferService" maxreceivedmessagesize="65536">
messageEncoding="Mtom"
transferMode="Buffered"
maxBufferSize="65536"
maxBufferPoolSize="524288"/>
</binding></basichttpbinding>
</bindings>
it works very fine but with a problem that i cant upload large files and then i realised that i have changed the maxReceivedMessageSize from 1GB to 64KB which made obvious to me.
and i change the side back to 1GB i get this exception
This factory buffers messages, so the message sizes must be in the range of an integer value.
Parameter name: bindingElement.MaxReceivedMessageSize
can anyway help me in solving my hindrances
thx in anticipation
wcf newbie
|
|
|
|
|
Can't really help, but you might want to contact the author of the article you were referring to. They might be able to shed some light.
"I guess it's what separates the professionals from the drag and drop, girly wirly, namby pamby, wishy washy, can't code for crap types." - Pete O'Hanlon
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
![Java | [Coffee]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/coffee.gif)
