Ttyjukl

validation vs test vs training accuracy, which one to compare for claiming overfit?

Confusion with the nameplate of an induction motor

Prove that the total distance is minimised (when travelling across the longest path)

Are there situations where a child is permitted to refer to their parent by their first name?

Potentiometer like component

Can the druid cantrip Thorn Whip really defeat a water weird this easily?

What is the blue range indicating on this manifold pressure gauge?

Provisioning profile doesn't include the application-identifier and keychain-access-groups entitlements

How is the Swiss post e-voting system supposed to work, and how was it wrong?

How do anti-virus programs start at Windows boot?

My story is written in English, but is set in my home country. What language should I use for the dialogue?

Is all copper pipe pretty much the same?

When were linguistics departments first established

Gravity alteration as extermination tool viable?

Can't remove a file with file mode bits a+rw

Block storage rewrites

Force user to remove USB token

Plywood subfloor won't screw down in a trailer home

Does splitting a potentially monolithic application into several smaller ones help prevent bugs?

US to Europe trip with Canada layover- is 52 minutes enough?

Is having access to past exams cheating and, if yes, could it be proven just by a good grade?

What is the definition of "Natural Selection"?

Touchscreen-controlled dentist office snowman collector game

What to do when during a meeting client people start to fight (even physically) with each others?

Deleting missing values from a dataset

How to execute JavaScipt on a webpage and then import the result on OSX?GeoLabels/Tooltips in Geographic Plots from DatasetImport a column of data, make a matrix from it and export it WITHOUT curly bracesShort way to query Dataset by the value of several columnsHow do I import a Dataset that contains lists?Going from Dataset back to underlying AssociationsElegant way to return first non-Missing column in DatasetHow to work with a Dataset?Best way to convert numerical columns of a dataset with missing values into a matrixWhat's the best way to import such dataset?

1

$begingroup$

I wanted to create a dataset of all UFO sightings in April of 2018. The
AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[

"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

"Data"}];

dataA2 = Flatten[Rest@dataA1, 1];

dataA3 = 

 Map[AssociationThread[First[dataA1], #] &, dataA2]

Dataset[dataA3]

import dataset associations web-access

share|improve this question

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths.
  $endgroup$
  – Carl Lange
  2 hours ago
  

  
  
  
  

add a comment | 

1

$begingroup$

I wanted to create a dataset of all UFO sightings in April of 2018. The
AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[

"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

"Data"}];

dataA2 = Flatten[Rest@dataA1, 1];

dataA3 = 

 Map[AssociationThread[First[dataA1], #] &, dataA2]

Dataset[dataA3]

import dataset associations web-access

share|improve this question

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths.
  $endgroup$
  – Carl Lange
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I wanted to create a dataset of all UFO sightings in April of 2018. The
AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[

"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

"Data"}];

dataA2 = Flatten[Rest@dataA1, 1];

dataA3 = 

 Map[AssociationThread[First[dataA1], #] &, dataA2]

Dataset[dataA3]

import dataset associations web-access

share|improve this question

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

$endgroup$

dataA1 = Import[

"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

"Data"}];

dataA2 = Flatten[Rest@dataA1, 1];

dataA3 = 

 Map[AssociationThread[First[dataA1], #] &, dataA2]

Dataset[dataA3]

share|improve this question

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

share|improve this question

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

edited 10 mins ago

Carl Lange

4,58311038

edited 10 mins ago

Carl Lange

4,58311038

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

asked 2 hours ago

Artem AnisimovArtem Anisimov

353

1 Answer
1

active

oldest

votes

4

$begingroup$

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[

  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

   "FullData"}]

dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]

dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];

Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

share|improve this answer

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately?
  $endgroup$
  – Artem Anisimov
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements.
  $endgroup$
  – Carl Lange
  26 mins ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "387"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f193199%2fdeleting-missing-values-from-a-dataset%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

4

$begingroup$

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[

  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

   "FullData"}]

dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]

dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];

Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

share|improve this answer

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately?
  $endgroup$
  – Artem Anisimov
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements.
  $endgroup$
  – Carl Lange
  26 mins ago
  

  
  
  
  

add a comment | 

4

$begingroup$

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[

  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

   "FullData"}]

dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]

dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];

Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

share|improve this answer

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately?
  $endgroup$
  – Artem Anisimov
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements.
  $endgroup$
  – Carl Lange
  26 mins ago
  

  
  
  
  

add a comment | 

$begingroup$

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[

  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

   "FullData"}]

dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]

dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];

Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

share|improve this answer

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

$endgroup$

dataA1 = Import[

  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 

   "FullData"}]

dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]

dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];

Dataset[dataA3]

share|improve this answer

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

answered 1 hour ago

Carl LangeCarl Lange

4,58311038

answered 1 hour ago

Carl LangeCarl Lange

4,58311038